Saturday, June 29, 2013

29/6/2013: Research Funding Does Not Seem to Match Research Performance

Very interesting:

From the abstract: "Agencies that fund scientific research must choose: is it more effective to give large grants to a few elite researchers, or small grants to many researchers? Large grants would be more effective only if scientific impact increases as an accelerating function of grant size."

So, the paper examines "the scientific impact of individual university-based researchers in three disciplines funded by the Natural Sciences and Engineering Research Council of Canada (NSERC)", based on "four indices of scientific impact:

  • numbers of articles published, 
  • numbers of citations to those articles, 
  • the most cited article, and 
  • the number of highly cited articles."
All of the above parameters are "measured over a four-year period" and referenced against "the amount of NSERC funding received".

Core findings:

  • "Impact is positively, but only weakly, related to funding" - which is disturbing, as it suggests that funding allocation is not academically efficient. 
  • "Researchers who received additional funds from a second federal granting council, the Canadian Institutes for Health Research, were not more productive than those who received only NSERC funding." Which suggests that the most important agency has trouble identifying and signalling by its grants allocation the academic 'winners'.
  • "Impact was generally a decelerating function of funding." Which is really bad. In basic terms, the more funds were pumped the less was the positive marginal impact. So that "impact per dollar was therefore lower for large grant-holders". Which "is inconsistent with the hypothesis that larger grants lead to larger discoveries". 
  • "Further, the impact of researchers who received increases in funding did not predictably increase." So obtaining a larger grant did not lead to subsequent improvement of the researcher output! 
  • "We conclude that scientific impact (as reflected by publications) is only weakly limited by funding."
  • And a big Boom! "We suggest that funding strategies that target diversity, rather than “excellence”, are likely to prove to be more productive." 
Now, the last point is what all funding agencies around the world are trying to avoid. Everywhere, the policy in funding research is on more concentration and on more winner-picking. Not on funding broader research and research groups.

Incidentally, if you are interested in this topic - what and how should be funded and prioritised in research and education - read my Sunday Times article tomorrow.


Anonymous said...

Interesting post. I think it also depends on what you mean by the 'effect' in'effective'. As a physicist, I would like to see more SFI funding spent on membership of international bodies such as CERN (so that many many students could undergo training at a world-class facility),before they go on to a variety of careers.
Rather this than giving large funding to a small numer of local projects that rarely result in the promised economic gain..
Regards, Cormac

TrueEconomics said...

Agree. The paper linked above defines the performance metrics etc. None, afair, are 'economic efficiency'-linked. Also, this is funding that excludes general funding for, say, memberships etc. But I do agree with you that more funding for membership in larger research programmes is probably a good way to spend funds.