This can be a serious problem with drug research. Suppose 5 researchers run 5 lidocaine tests and 1 finds that it makes epilepsy slightly worse, 3 find that it doesn't do much of anything, and 1 finds that it makes it slightly better. Guess which paper is more likely to get published?
In a study of whether bilingualism makes you smarter (it doesn't), Angela de Bruin came up with a clever way to get past part of the bias: look at conference proceedings instead.
The rationale was straightforward: conferences are places where people present in-progress research. They report on studies that they are running, initial results, initial thoughts. If there were a systematic bias in the field against reporting negative results—that is, results that show no effects of bilingualism—then there should be many more findings of that sort presented at conferences than actually become published.
That’s precisely what de Bruin found. At conferences, about half the presented results provided either complete or partial support for the bilingual advantage on certain tasks, while half provided partial or complete refutation. When it came to the publications that appeared after the preliminary presentation, though, the split was decidedly different. Sixty-eight per cent of the studies that demonstrated a bilingual advantage found a home in a scientific journal, compared to just twenty-nine per cent of those that found either no difference or a monolingual edge.
Kudos to Angela!