nep-sog New Economics Papers
on Sociology of Economics
Issue of 2022‒12‒05
four papers chosen by
Jonas Holmström
Axventure AB
  1. We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell us about 𝒑-Hacking and Publication Bias in Online Experiments By Brodeur, Abel; Cook, Nikolai; Heyes, Anthony
  2. Top 25% Institutions and Economists in Viet Nam, as of September 2022 By St. Louis, ResDiv FED
  3. Top 25% Institutions and Economists in Viet Nam, as of October 2022 By , 太公釣魚
  4. Auditing the Auditors: An evaluation of the REF2021 Output Results By Linton, O. B.; Xu, E.
  1. By: Brodeur, Abel; Cook, Nikolai; Heyes, Anthony
    Abstract: Amazon's Mechanical Turk is a very widely-used tool in business and economics research, but how trustworthy are results from well-published studies that use it? Analyzing the universe of hypotheses tested on the platform and published in leading journals between 2010 and 2020 we find evidence of widespread p-hacking, publication bias and over-reliance on results from plausibly under-powered studies. Even ignoring questions arising from the characteristics and behaviors of study recruits, the conduct of the research community itself erodes substantially the credibility of these studies' conclusions. The extent of the problems vary across the business, economics, management and marketing research fields (with marketing especially afflicted). The problems are not getting better over time and are much more prevalent than in a comparison set of non-online experiments. We explore correlates of increased credibility.
    Keywords: online crowd-sourcing platforms,Amazon Mechanical Turk,p-hacking,publication bias,statistical power,research credibility
    JEL: B41 C13 C40 C90
    Date: 2022
    URL: http://d.repec.org/n?u=RePEc:zbw:i4rdps:8&r=sog
  2. By: St. Louis, ResDiv FED
    Abstract: The rankings: Top 25% institutions in Viet Nam, all authors, all publication years. For Viet Nam, there are 201 authors affiliated with 57 institutions.
    Date: 2022–10–11
    URL: http://d.repec.org/n?u=RePEc:osf:osfxxx:vzs45&r=sog
  3. By: , 太公釣魚
    Abstract: The IDEAS RePEc rankings: Top 25% institutions in Viet Nam, all authors, all publication years. For Viet Nam, there are 201 authors affiliated with 57 institutions. IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis. (November 7, 2022)
    Date: 2022–11–06
    URL: http://d.repec.org/n?u=RePEc:osf:osfxxx:tbwr7&r=sog
  4. By: Linton, O. B.; Xu, E.
    Abstract: We apply the Hole algorithm to evaluate the REF2021 output quality exercise. We find that the implied journal ranking agrees quite closely with the ABS-SCOB journal ranking, and in particular the GPA’s agree with a 91% correlation.
    Keywords: Journal quality, Ranking, Research funding
    JEL: A10
    Date: 2022–11–08
    URL: http://d.repec.org/n?u=RePEc:cam:camdae:2266&r=sog
This nep-sog issue is ©2022 by Jonas Holmström. It is provided as is without any express or implied warranty. It may be freely redistributed in whole or in part for any purpose. If distributed in part, please include this notice.
General information on the NEP project can be found at http://nep.repec.org. For comments please write to the director of NEP, Marco Novarese at <director@nep.repec.org>. Put “NEP” in the subject, otherwise your mail may be rejected.
NEP’s infrastructure is sponsored by the School of Economics and Finance of Massey University in New Zealand.
This page is being maintained by Thomas Krichel. It was last updated on 2023‒07‒08 at 15:12.