29/6/2013: Research Funding Does Not Seem to Match Research Performance

Very interesting:

http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0065263

From the abstract: "Agencies that fund scientific research must choose: is it more effective to give large grants to a few elite researchers, or small grants to many researchers? Large grants would be more effective only if scientific impact increases as an accelerating function of grant size."

So, the paper examines "the scientific impact of individual university-based researchers in three disciplines funded by the Natural Sciences and Engineering Research Council of Canada (NSERC)", based on "four indices of scientific impact:

• numbers of articles published, 
• numbers of citations to those articles, 
• the most cited article, and 
• the number of highly cited articles."

All of the above parameters are "measured over a four-year period" and referenced against "the amount of NSERC funding received".

Core findings:

• "Impact is positively, but only weakly, related to funding" - which is disturbing, as it suggests that funding allocation is not academically efficient. 
• "Researchers who received additional funds from a second federal granting council, the Canadian Institutes for Health Research, were not more productive than those who received only NSERC funding." Which suggests that the most important agency has trouble identifying and signalling by its grants allocation the academic 'winners'.
• "Impact was generally a decelerating function of funding." Which is really bad. In basic terms, the more funds were pumped the less was the positive marginal impact. So that "impact per dollar was therefore lower for large grant-holders". Which "is inconsistent with the hypothesis that larger grants lead to larger discoveries". 
• "Further, the impact of researchers who received increases in funding did not predictably increase." So obtaining a larger grant did not lead to subsequent improvement of the researcher output! 
• "We conclude that scientific impact (as reflected by publications) is only weakly limited by funding."
• And a big Boom! "We suggest that funding strategies that target diversity, rather than “excellence”, are likely to prove to be more productive." 

Now, the last point is what all funding agencies around the world are trying to avoid. Everywhere, the policy in funding research is on more concentration and on more winner-picking. Not on funding broader research and research groups.

Incidentally, if you are interested in this topic - what and how should be funded and prioritised in research and education - read my Sunday Times article tomorrow.