Monday, November 16, 2015

Riskless science

Information costs money. Groups that fund research sometimes go for radical projects, but generally they play it safe.

There are some good reasons for this. The article says

That leads to a large duplication of research and makes it slower for the whole field to move forward.
Evans and his colleagues looked at the networks of knowledge in biomedical medicine papers and 30,000 patents spanning more than 30 years. They analyzed the relationships between medically useful molecules and found that as a field matures, scientists are more likely to study closely related molecules rather than distantly related ones.
"You'd expect them go the opposite direction: At first, picking important things but then there are diminishing returns in those experiments, so over time they'd shift to more risky experiments, choosing things further from each other in the network," said Evans. But that's not what happened.

Duplication of experiments helps keep science correct, and looking at related topics helps cross-check too. "If you look at 100 histograms, generally one of them will have a 3-sigma peak somewhere," just from random variations. If you publish the 3-sigma excess, you've screwed up. Further studies are needed. Also, at some point you are the expert on the research, and people expect you to be able to answer the questions about it. You've a good track record of finding things out, and so are a better candidate for scarce research dollars.

Sort of--as the article says. After a while you get diminishing returns, and the question is do the grant agencies get more bang for the buck funding a sure thing with tiny payback, or a long shot with a bigger payback? The author thinks the agencies and the gatekeepers skew too conservative.

In my field the experiments are very expensive, and a proposal can go through many years of review before the funders that be are satisfied you can deliver something new and interesting. And they check the budget too: Will we get more from this $1,000,000 project or from 5 $200,000 ones? One recent proposal some colleagues submitted was agreed to be novel and useful, but the main result was limited, and funding it would exclude other excellent proposals. That's the way it goes... the overall science budget isn't going to grow while there's a war on.

A better way to fund science might be giving money to investigators instead of projects, as the Howard Hughes Medical Institute does, Evans said. In the heyday of Bell Labs, funding was given to groups rather than individual scientists – and efforts like these move some of the incentives to let scientists take on riskier experiments.

That might work in biomedical, but when an experiment takes 8 years and requires a few hundred people...

When I first started exploring an interesting symmetry, I would have guessed it was 10 to 1 against it having physics relevance. Now I'd guess 100 to 1 against. Unfortunately, the odds of it being a usefully fundamental mathematical breakthrough are also probably 100:1 against, so this is probably just an odd corner that I work on in my "copious free time." At those odds, would you fund my salary for half a year to explore it? ... Didn't think so. But so long as I'm willing to take decades to solve a problem that an intense year would solve, I can take whatever risks I please. Just don't ask for answers this year.

No comments: