|
on Sociology of Economics |
| By: | Brodeur, Abel; Cook, Nikolai; Heyes, Anthony |
| Abstract: | Amazon Mechanical Turk is a very widely-used tool in business and economics research, but how trustworthy are results from well-published studies that use it? Analyzing the universe of hypotheses tested on the platform and published in leading journals between 2010 and 2020 we find evidence of widespread p-hacking, publication bias and over-reliance on results from plausibly under-powered studies. Even ignoring questions arising from the characteristics and behaviors of study recruits, the conduct of the research community itself erode substantially the credibility of these studies' conclusions. The extent of the problems vary across the business, economics, management and marketing research fields (with marketing especially afflicted). The problems are not getting better over time and are much more prevalent than in a comparison set of non-online experiments. We explore correlates of increased credibility. |
| Date: | 2022–08–11 |
| URL: | http://d.repec.org/n?u=RePEc:osf:metaar:a9vhr&r=sog |
| By: | Brodeur, Abel; Cook, Nikolai; Hartley, Jonathan; Heyes, Anthony |
| Abstract: | Randomized controlled trials (RCTs) are increasingly prominent in economics, with pre-registration and pre-analysis plans (PAPs) promoted as important in ensuring the credibility of findings. We investigate whether these tools reduce the extent of p-hacking and publication bias by collecting and studying the universe of test statistics, 15,992 in total, from RCTs published in 15 leading economics journals from 2018 through 2021. In our primary analysis, we find no meaningful difference in the distribution of test statistics from pre-registered studies, compared to their non-pre-registered counterparts. However, pre-registered studies that have a complete PAP are significantly less p-hacked. These results point to the importance of PAPs, rather than pre-registration in itself, in ensuring credibility. |
| Date: | 2022–08–11 |
| URL: | http://d.repec.org/n?u=RePEc:osf:metaar:uxf39&r=sog |
| By: | Röseler, Lukas (University of Bamberg); Gendlina, Taisia; Krapp, Josefine; Labusch, Noemi; Schütz, Astrid (University of Bamberg) |
| Abstract: | A considerable proportion of psychological research has not been replicable, and estimates range from 9% to 77% for nonreplicable results. The extent to which vast proportions of studies in the field are replicable is still unknown, as researchers lack incentives for publishing individual replication studies. When preregistering replication studies via the Open Science Foundation website (OSF, osf.io), researchers can publicly register their results without having to publish them and thus circumvent file-drawer effects. We analyzed data from 139 replication studies for which the results were publicly registered on the OSF and found that out of 62 reports that included the authors’ assessments, 23 were categorized as “informative failures to replicate” by the original authors. 24 studies allowed for comparisons between the original and replication effect sizes, and whereas 75% of the original effects were statistically significant, only 30% of the replication effects were. The replication effects were also significantly smaller than the original effects (approx. 38% the size). Replication closeness did not moderate the difference between the original and the replication effects. Our results provide a glimpse into estimating replicability for studies from a wide range of psychological fields chosen for replication by independent groups of researchers. We invite researchers to browse the Replication Database (ReD) ShinyApp, which we created to check for whether seminal studies from their respective fields have been replicated. Our data and code are available online: https://osf.io/9r62x/ |
| Date: | 2022–08–16 |
| URL: | http://d.repec.org/n?u=RePEc:osf:metaar:8psw2&r=sog |
| By: | Timothy Hunt; Hamish Low; Sarah Smith |
| Abstract: | This note analyses how the allocation of research funding across disciplines in 2022 came about following the UK Research Excellence Framework (REF). There are two, related issues. First, REF2021 led to substantial changes in funding allocations across disciplines, particularly in Social Sciences. Second, economics emerges as the biggest, relative loser in terms of total funding |
| Date: | 2022–10–19 |
| URL: | http://d.repec.org/n?u=RePEc:oxf:wpaper:986&r=sog |
| By: | POP, NAPOLEON (National Institute for Economic Research - Romanian Academy); IOAN-FRANC, VALERIU (National Institute for Economic Research - Romanian Academy) |
| Abstract: | The academic world says that the divorce between economists and politicians became visible during the financial crisis that began in 2008, when the economists were criticised for failing to preview the crisis, on the one hand, and, on the other hand, for the slowness in accompanying the solutions to remedy the crisis which should have been suggested by the politicians in an emergency situation, of financial collapse. The divorce itself has been lasting for a long time with the relationship deteriorating for many other reasons. In the background there is (1) an elite of Nobel prize winning economists with permanent training and inclination towards research and results worthy of consideration, and (2) a political class in a quasi-constant professional decline. It is a worrying fact, also publicly acknowledged. The 21st century, which marks the greatest technological advance of the fourth industrial revolution, started with avatars coming seemingly from nowhere. Yet, this century which began with so much hope, brings us in the face of yet another break between science and practice. The phenomenon we refer to in the current study is just a small part of what it should have been the collaboration between politicians and science in general, but the specifics of the divorce we approach in this essay is the fact that it deepens. |
| Keywords: | general economics, information and uncertainty, economic policies, politics and sociological system, national government policies, public policy |
| JEL: | A1 A11 B20 B52 H5 J18 |
| Date: | 2022–11 |
| URL: | http://d.repec.org/n?u=RePEc:ror:wpince:221116&r=sog |