| Abstract: |
We estimate the impact of participating in the NZ Marsden Fund on research
output trajectories, by comparing the subsequent performance of funded
researchers to those who submitted proposals but were not funded. We control
for selection bias using the evaluations of the proposals generated by the
grant selection process. We carry out the analysis in two data frames. First
we consider the researcher teams behind 1263 second-round proposals submitted
2003-2008, and look at the post-proposal publication and citation performance
of the team as a whole, as a function of pre-proposal performance, the ranking
of the proposal by the panel, and the funding. This estimation does not deal
with individual researchers’ multiple proposals and funding over time. To
disentangle these effects, we consider the 1500 New Zealand researchers who
appeared on any of these proposals, and estimate a model predicting annual
individual performance as a function of previous performance, recent proposal
activity, ranking of any recent proposals, and funding received through recent
proposals. Overall, we find that funding is associated with a 6-15% increase
in publications and a 22-26% increase in citation-weighted papers for research
teams. For individuals, funding is associated with a 3-5% increase in annual
publications, and a 5-8% increase in citation-weighted papers for 5 years
after grant; however, the lag structure and persistence of this effect
post-grant is difficult to pin down. Surprisingly, we find no systematic
evidence that the evaluation of proposals by the Marsden system is predictive
of subsequent success. We conclude that the Marsden Fund is modestly
successful in increasing scientific performance, but that the selection
process does not appear to be effective in discriminating among second-round
proposals in terms of their likely success. |