|
on Sociology of Economics |
| By: | Carrell, Scott; Figlio, David; Lusher, Lester |
| Abstract: | We study how author-editor and author-reviewer network connectivity and "match" influence editor decisions and reviewer recommendations of economic research at the Journal of Human Resources. Our empirical strategy employs several dimensions of fixed effects to overcome concerns of endogenous assignment of papers to editors and reviewers. Authors who attended the same PhD program, were ever colleagues with, are affiliates of the same National Bureau of Economic Research program(s), or are more closely linked via coauthorship networks as the handling editor are significantly more likely to avoid a desk rejection. Likewise, authors from the same PhD program or who previously worked with the reviewer are significantly more likely to receive a positive evaluation. We also find that sharing "signals" of ability, such as publishing in the "top five", attending a high ranked PhD program, or being employed by a similarly ranked economics department, significantly influences editor decisions and/or reviewer recommendations. We find some evidence that published papers with greater author-editor connectivity subsequently receive fewer citations. |
| Date: | 2023 |
| URL: | http://d.repec.org/n?u=RePEc:zbw:i4rdps:60&r=sog |
| By: | Ankel-Peters, Jörg; Fiala, Nathan; Neubauer, Florian |
| Abstract: | This paper reviews the impact of replications published as comments in the American Economic Review between 2010 and 2020. We examine their citations and influence on the original papers' subsequent citations. Our results show that comments are barely cited, and they do not affect the original paper's citations - even if the comment diagnoses substantive problems. Furthermore, we conduct an opinion survey among replicators and authors and find that there often is no consensus on whether the original paper's contribution sustains. We conclude that the economics literature does not self-correct, and that robustness and replicability are hard to define in economics. |
| Keywords: | replication, citations, meta-science |
| JEL: | A11 A14 B40 |
| Date: | 2023 |
| URL: | http://d.repec.org/n?u=RePEc:zbw:i4rdps:68&r=sog |
| By: | Van-Thien Nguyen; René Carraz |
| Abstract: | This paper proposes a new method for matching patents with academic publications to create patent-paper pairs (PPP). These pairs can identify instances where a research result is both applied in a patent and published in a paper. The study focuses on a sample of top research-intensive universities and laboratories in Japan, utilizing a new dataset that contains patent-to-article citations and a machine learning model as part of the matching process. Expert consultations were conducted to enhance the robustness of the methodology. Focusing on a set of 14 Japanese universities and 3 national research laboratories, using patent (USPTO) and publication data (OpenAlex) between 1998 and 2018, we built a dataset of 3, 177 PPPs out of 7, 766 granted patents and 91, 213 publications. The results demonstrate that this phenomenon is widespread in academia and our data show the diversity of the academic disciplines and technical field involved, highlighting the intricate connections between scientific and technical concepts and communities. On the methodological side, we documented in-depth complementary validation techniques to enhance the precision and reliability of our matching algorithm. Using open-source data, our methodology is adaptable to diverse national contexts and can be readily adopted by other research teams investigating similar topics. |
| Keywords: | Patent Paper Pair; Methodology; Matching algorithm; Academic patent; Japan. |
| Date: | 2023 |
| URL: | http://d.repec.org/n?u=RePEc:ulp:sbbeta:2023-29&r=sog |
| By: | Slichter, David; Tran, Nhan |
| Abstract: | Are estimates typically closer to the true parameter value when those estimates are published in highly-ranked economics journals? Using 14, 387 published estimates from 24 large literatures, we find that, within literatures, the mean and variance of parameter estimates have little or no correlation with journal rank. Therefore, regardless of what the true parameter value is that a literature is attempting to estimate, it cannot be that estimates in higher-ranked journals are on average noticeably closer to it. We discuss possible explanations and implications. |
| Keywords: | Meta-analysis, scientific methods, publication, science of science |
| JEL: | A11 C13 C18 |
| Date: | 2023–05 |
| URL: | http://d.repec.org/n?u=RePEc:pra:mprapa:118433&r=sog |
| By: | Hüttel, Silke; Hess, Sebastian |
| Abstract: | The scientific production system is of ultimate importance for the way how humans address global challenges. Recently, scholars have begun to voice concerns about structural inefficiencies within this system, as e.g. the replication crisis, the p-value debate or the identification of various forms of publication bias have brought up. Most suggested remedies tend to address only partial aspects of the system's inefficiencies, while no unifying agenda towards an overall transformation of the system has yet emerged. We argue that a unifying agenda is even more urgently needed in light of Artificial intelligence (AI) that is arising as a tool for scientific writing services. Without appropriate reactions from the Q science community, this trend may even exponentiate present credibility problems due to limited replicability and ritual-based statistical practice, while amplifying all forms of already existing biases. Our review of these developments suggests that näive openness in the science system alone will unlikely lead to major efficiency gains. We contribute by identifying key elements for the definition of transformation pathways towards open, democratic and conscious learning, teaching, reviewing and publishing that will be supported by openly maintained AI tools. As part of this transition, roles and incentives for reviewers will have to gain in relation to authors: Future Q scientists will have to write less, learn differently and review more. |
| Date: | 2023 |
| URL: | http://d.repec.org/n?u=RePEc:zbw:daredp:2302&r=sog |
| By: | Kevin Lang |
| Abstract: | When economists analyze a well-conducted RCT or natural experiment and find a statistically significant effect, they conclude the null of no effect is unlikely to be true. But how frequently is this conclusion warranted? The answer depends on the proportion of tested nulls that are true and the power of the tests. I model the distribution of t-statistics in leading economics journals. Using my preferred model, 65% of narrowly rejected null hypotheses and 41% of all rejected null hypotheses with |t| |
| JEL: | A10 C12 |
| Date: | 2023–09 |
| URL: | http://d.repec.org/n?u=RePEc:nbr:nberwo:31666&r=sog |