nep-sog 2024-02-05 papers

on Sociology of Economics

Issue of 2024‒02‒05
two papers chosen by
Jonas Holmström, Axventure AB

Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias? Evidence from 15, 992 Test Statistics and Suggestions for Improvement

By:	Brodeur, Abel; Cook, Nikolai M.; Hartley, Jonathan S.; Heyes, Anthony
Abstract:	Pre-registration is regarded as an important contributor to research credibility. We investigate this by analyzing the pattern of test statistics from the universe of randomized controlled trials (RCT) studies published in 15 leading economics journals. We draw two conclusions: (a) Pre-registration frequently does not involve a pre-analysis plan (PAP), or sufficient detail to constrain meaningfully the actions and decisions of researchers after data is collected. Consistent with this, we find no evidence that pre-registration in itself reduces p-hacking and publication bias. (b) When pre-registration is accompanied by a PAP we find evidence consistent with both reduced phacking and publication bias.
Keywords:	Pre-analysis plan, Pre-registration, p-Hacking, Publication, bias, Research credibility
JEL:	B41 C13 C40 C93
Date:	2024
URL:	http://d.repec.org/n?u=RePEc:zbw:i4rdps:101&r=sog

Promoting Reproducibility and Replicability in Political Science

By:	Brodeur, Abel; Esterling, Kevin; Ankel-Peters, Jörg; Bueno, Natália S.; Desposato, Scott; Dreber, Anna; Genovese, Federica; Green, Donald P.; Hepplewhite, Matthew; Hoces de la Guardia, Fernando; Johannesson, Magnus; Kotsadam, Andreas; Miguel, Edward; Velez, Yamil R.; Young, Lauren
Abstract:	This article reviews and summarizes current reproduction and replication practices in political science. We first provide definitions for reproducibility and replicability. We then review data availability policies for 28 leading political science journals and present the results from a survey of editors about their willingness to publish comments and replications. We discuss new initiatives that seek to promote and generate highquality reproductions and replications. Finally, we make the case for standards and practices that may help increase data availability, reproducibility, and replicability in political science.
Keywords:	Reproducibility, replicability, political science
Date:	2024
URL:	http://d.repec.org/n?u=RePEc:zbw:i4rdps:100&r=sog

This nep-sog issue is ©2024 by Jonas Holmström. It is provided as is without any express or implied warranty. It may be freely redistributed in whole or in part for any purpose. If distributed in part, please include this notice.

General information on the NEP project can be found at https://nep.repec.org. For comments please write to the director of NEP, Marco Novarese at <director@nep.repec.org>. Put “NEP” in the subject, otherwise your mail may be rejected.

NEP’s infrastructure is sponsored by the School of Economics and Finance of Massey University in New Zealand.