How economic theory and the Netflix Prize could make research funding more efficient

Two scientists believe that, with professors vying for such a small pool of funds, the grant-application process has become a competition not over who has the best ideas, but who is the best at writing grant applications. In a paper published Jan. 2 in the journal PLOS Biology, co-authors Carl Bergstrom, a professor of biology at the University of Washington, and Kevin Gross, a professor of statistics at North Carolina State University, use the economic theory of contests to illustrate how this competitive system has made the pursuit of research funding inefficient and unsustainable. They show that alternative methods, such as a partial lottery to award grants, could help get professors back in the lab where they belong.

To receive a grant today, professors apply to funding agencies like the National Science Foundation or the National Institutes of Health. Reviewers evaluate and rank the applications, and the highest-ranking applications receive grant funding.

But over time, the percentage of proposals that receive funding has dropped dramatically. This is largely because the pool of available funds has not grown to keep pace with the number of STEM researchers.

"Back in the 1970s, the top 40 to 50 percent of applications to agencies were funded," said Bergstrom. "Agencies merely had to separate the good research plans from the bad based on the grant applications."

Funding thresholds for grant applications have tightened steadily since the 1970s. In 2003, only the top 20 percent of research project grant applications to the National Institute of Allergy & Infectious Diseases were funded. In 2013, the success rate had plummeted to 8 percent. Gross and Bergstrom argue that the funding pool has grown so small relative to the number of applicants that the nature of the grant-application process had changed.

"When agencies only fund the top 10 or 20 percent, they aren't just separating bad ideas from good ideas," said Bergstrom. "They're also separating good from good."

"This has two effects on the grant-application process," said Gross. "First, professors must apply for more and more grants before they're awarded one. Second, the application process becomes a contest to determine who can write the best grant proposals -- so professors spend more and more time trying to perfect each individual application."

Gross and Bergstrom realized that today's grant-application process can be described using the economic theory of contests. In contest theory, teams compete to produce a product or complete a task for an agency; the agency picks a winner and retains the fruits of the team's efforts, while the winning team receives a prize such as cash. For the Netflix Prize, for example, teams competed to produce an algorithm that would predict how users would rank films on its streaming service. Netflix received the winning algorithm, while the winning team pocketed $1 million.

"If we were to apply contest theory to grants, then professors are the ones competing to create a product -- the best grant application -- for the agency," said Gross. "That's not a particularly good system, though, because the funding agency doesn't want grant applications for their own sake. They want to fund research."

In their paper, Bergstrom and Gross illustrate how the grant-application process is consistent with economic contest models. They show how funding a relatively small fraction of grant applications -- such as the top 10 or 15 percent -- makes the practice of science inefficient: The negative costs associated with trying to produce the best grant application could potentially outweigh the economic value of the science produced.

If agencies funded a higher percentage of applications, professors could spend less time trying to write the perfect grant application. In addition, funding agencies wouldn't have to subjectively choose winners among high-quality proposals that are all based on sound science. But this option would require significantly expanding funding to agencies like the NIH and the NSF, a politically difficult task.

Using the economic theory of contests, Gross and Bergstrom modeled a controversial alternative: awarding grants instead by partial lottery. Under a partial lottery system, funds are awarded by random draw among a pool of high-ranking grants -- the top 40 percent, for example. Since applicants would be aiming to clear a lower bar for a smaller prize -- a shot at the lottery instead of a guaranteed payout for winning proposals -- the contest theory model predicts that applicants would spend less time trying to perfect their applications, Bergstrom said.

Partial lotteries have been proposed by others, such as UW professor of laboratory medicine Ferric Fang and Johns Hopkins professor Arturo Casadevall. They're also used by two funding agencies in New Zealand and the Volkswagen Foundation. Gross and Bergstrom simply use contest theory to show how this system could also free professors from the seemingly endless cycle of grant applications.

But partial lotteries aren't the only viable solution, they say. Funding agencies could also award grants based on merit, such as a professor's past record of excellence in research. But that system also would need mechanisms to help early-career faculty and professors from underrepresented groups obtain grants, Bergstrom said. Hybrid systems are another option, such as a partial lottery for early-career faculty and merit-based grants for later-career faculty.

"There are many potential routes out of the current hole," said Bergstrom. "What doesn't change is our conclusion that the current grant-application system is fundamentally inefficient and unsustainable."