nep-sog New Economics Papers
on Sociology of Economics
Issue of 2013‒03‒23
one paper chosen by
Jonas Holmström
Swedish School of Economics and Business Administration

  1. Star Wars: The Empirics Strike Back By Brodeur, Abel; Lé, Mathias; Sangnier, Marc; Zylberberg, Yanos

  1. By: Brodeur, Abel (Paris School of Economics); Lé, Mathias (Paris School of Economics); Sangnier, Marc (University of Aix-Marseille II); Zylberberg, Yanos (CREI and Universitat Pompeu Fabra)
    Abstract: Journals favor rejection of the null hypothesis. This selection upon tests may distort the behavior of researchers. Using 50,000 tests published between 2005 and 2011 in the AER, JPE, and QJE, we identify a residual in the distribution of tests that cannot be explained by selection. The distribution of p-values exhibits a camel shape with abundant p-values above 0.25, a valley between 0.25 and 0.10 and a bump slightly below 0.05. The missing tests (with p-values between 0.25 and 0.10) can be retrieved just after the 0.05 threshold and represent 10% to 20% of marginally rejected tests. Our interpretation is that researchers might be tempted to inflate the value of those almost-rejected tests by choosing a "significant" specification. We propose a method to measure inflation and decompose it along articles' and authors' characteristics.
    Keywords: hypothesis testing, distorting incentives, selection bias, research in economics
    JEL: A11 B41 C13 C44
    Date: 2013–03

This nep-sog issue is ©2013 by Jonas Holmström. It is provided as is without any express or implied warranty. It may be freely redistributed in whole or in part for any purpose. If distributed in part, please include this notice.
General information on the NEP project can be found at For comments please write to the director of NEP, Marco Novarese at <>. Put “NEP” in the subject, otherwise your mail may be rejected.
NEP’s infrastructure is sponsored by the School of Economics and Finance of Massey University in New Zealand.