| Abstract: |
Science funding agencies such as the NIH, NSF, and their counterparts around
the world are often criticized for being too conservative, funding incremental
innovations over more radical but riskier projects. One explanation for their
conservatism is the way the agencies use peer review of scientific proposals.
Peer review is the cornerstone of research allocation decisions, but agencies
typically base decisions on a simple average of peer review scores. More novel
ideas are less likely to gain consistently high ratings across evaluators and
are less likely to be funded. Using a discrete choice experiment conducted
with a large sample of active biomedical researchers, we find that—in contrast
to funding agencies—scientists systematically prefer to fund projects with
more reviewer dissensus. Rather than purely focusing on the first moment of
the distribution of reviewer scores, they also value the second moment.
Further, scientists with the greatest domain expertise on a proposal are more
enthusiastic about dissensus, and while appetite for dissensus shrinks as
budgets become tighter, it does not disappear completely. Applying our
estimates to prior studies mimicking NIH’s review process shows that
incorporating expert scientists’ preferences for dissensus would change
marginal funding decisions for ten percent of projects worth billions of
dollars per year. |