|
on Sociology of Economics |
| By: | Christopher Snyder; Ran Zhuo |
| Abstract: | The increasing demand for rigor in empirical economics has led to the growing use of auxiliary tests (balance, specification, over-identification, placebo, etc.) supporting the credibility of a paper's main results. We dub these "sniff tests" because standards for passing are subjective and rejection is bad news for the author. Sniff tests offer a new window into publication bias since authors prefer them to be insignificant, the reverse of standard statistical tests. Collecting a sample of nearly 30,000 sniff tests across 60 economics journals, we provide the first estimate of their aggregate probability-value (p-value) distribution. For the subsample of balance tests in randomized controlled trials (for which the distribution of p-values is known to be uniform absent publication bias, allowing reduced-form methods to be employed) estimates suggest that 45% of failed tests remain in the "file drawer" rather than being published. For the remaining sample with an unknown distribution of p-values, structural estimates suggest an even larger file-drawer problem, as high as 91%. Fewer significant sniff tests show up in top-tier journals, smaller tables, and more recent articles. We find no evidence of author manipulation other than a tendency to overly attribute significant sniff tests to bad luck. |
| JEL: | A14 B41 C18 |
| Date: | 2018–09 |
| URL: | http://d.repec.org/n?u=RePEc:nbr:nberwo:25058&r=sog |
| By: | Brodeur, Abel (University of Ottawa); Cook, Nikolai (University of Ottawa); Heyes, Anthony (University of Ottawa) |
| Abstract: | The economics 'credibility revolution' has promoted the identification of causal relationships using difference-in-differences (DID), instrumental variables (IV), randomized control trials (RCT) and regression discontinuity design (RDD) methods. The extent to which a reader should trust claims about the statistical significance of results proves very sensitive to method. Applying multiple methods to 13,440 hypothesis tests reported in 25 top economics journals in 2015, we show that selective publication and p-hacking is a substantial problem in research employing DID and (in particular) IV. RCT and RDD are much less problematic. Almost 25% of claims of marginally significant results in IV papers are misleading. |
| Keywords: | research methods, causal inference, p-curves, p-hacking, publication bias |
| JEL: | A11 B41 C13 C44 |
| Date: | 2018–08 |
| URL: | http://d.repec.org/n?u=RePEc:iza:izadps:dp11796&r=sog |
| By: | María Victoria Anauati; Ramiro H. Gálvez; Sebastian Galiani |
| Abstract: | Economics places a strong emphasis on publishing in a narrow set of top tier journals. Given that venue reputation does not necessarily go hand in hand with citation performance, we study how citation patterns differ across journal tiers (top five, second tier, and top field). By analyzing citations of 6,083 articles, we find that citation patterns effectively vary greatly across tiers, affecting not only articles’ total citations but also their distribution through time (i.e., their life cycles). Moreover, the way patterns differ across tiers is strongly associated to articles’ success (measured by citation counts) and fields of economics research. |
| JEL: | A1 |
| Date: | 2018–09 |
| URL: | http://d.repec.org/n?u=RePEc:nbr:nberwo:25101&r=sog |
| By: | James J. Heckman (The University of Chicago); Sidharth Moktan (Center for the Economics of Human Development, University of Chicago) |
| Abstract: | This paper examines the relationship between placement of publications in Top Five (T5) journals and receipt of tenure in academic economics departments. Analyzing the job histories of tenure-track economists hired by the top 35 U.S. economics departments, we find that T5 publications have a powerful influence on tenure decisions and rates of transition to tenure. A survey of the perceptions of young economists supports the formal statistical analysis. Pursuit of T5 publications has become the obsession of the next generation of economists. However, the T5 screen is far from reliable. A substantial share of influential publications appear in non-T5 outlets. Reliance on the T5 to screen talent incentivizes careerism over creativity. |
| Keywords: | tenure and promotion practices, career concerns, economics publishing, citations |
| JEL: | A14 I23 J44 O31 |
| Date: | 2018–09 |
| URL: | http://d.repec.org/n?u=RePEc:hka:wpaper:2018-070&r=sog |
| By: | Fulvio Castellacci (TIK Centre, University of Oslo); Clara Viñas-Bardolet (TIK Centre, University of Oslo) |
| Abstract: | Temporary contracts are increasingly used in academia. This is a major concern for non-tenured researchers, since weak job security may hamper job satisfaction. In spite of the relevance of this topic, scholarly research on the theme is scant. This paper presents an empirical analysis of the role of academic tenure for job satisfaction of researchers in European countries. The work uses data from the MORE2 survey, a recent large-scale representative survey of researchers in all European countries. The results show that, ceteris paribus, academics with a permanent contract are on average more satisfied with their job than those that are employed on a temporary basis. We also show that academic tenure is a relatively more important factor of job satisfaction for researchers at an intermediate stage of the career. Finally, we point out some important differences in the working of the model among European countries. Our hypotheses receive significant empirical support for the groups of Continental EU and Nordic economies, which combine high job satisfaction and good working conditions, on the one hand, with relatively weak job security for younger academics, on the other. |
| Date: | 2018–10 |
| URL: | http://d.repec.org/n?u=RePEc:tik:inowpp:20181008&r=sog |
| By: | Susan Athey; Michael Luca |
| Abstract: | As technology platforms have created new markets and new ways of acquiring information, economists have come to play an increasingly central role in tech companies – tackling problems such as platform design, strategy, pricing, and policy. Over the past five years, hundreds of PhD economists have accepted positions in the technology sector. In this paper, we explore the skills that PhD economists apply in tech companies, the companies that hire them, the types of problems that economists are currently working on, and the areas of academic research that have emerged in relation to these problems. |
| JEL: | A11 D4 L0 M2 |
| Date: | 2018–09 |
| URL: | http://d.repec.org/n?u=RePEc:nbr:nberwo:25064&r=sog |