|
on Sociology of Economics |
| By: | Stelmakh, Ivan; Rastogi, Charvi; Liu, Ryan; Chawla, Shuchi; Shah, Nihar; Echenique, Federico |
| Abstract: | Citations play an important role in researchers careers as a key factor in evaluation of scientific impact. Many anecdotes advice authors to exploit this fact and cite prospective reviewers to try obtaining a more positive evaluation for their submission. In this work, we investigate if such a citation bias actually exists: Does the citation of a reviewers own work in a submission cause them to be positively biased towards the submission? In conjunction with the review process of two flagship conferences in machine learning and algorithmic economics, we execute an observational study to test for citation bias in peer review. In our analysis, we carefully account for various confounding factors such as paper quality and reviewer expertise, and apply different modeling techniques to alleviate concerns regarding the model mismatch. Overall, our analysis involves 1, 314 papers and 1, 717 reviewers and detects citation bias in both venues we consider. In terms of the effect size, by citing a reviewers work, a submission has a non-trivial chance of getting a higher score from the reviewer: an expected increase in the score is approximately 0.23 on a 5-point Likert item. For reference, a one-point increase of a score by a single reviewer improves the position of a submission by 11% on average. |
| Keywords: | Humans, Prospective Studies, Peer Review, Bias, Research Personnel, Machine Learning, Peer Review, Research |
| Date: | 2023–01–01 |
| URL: | http://d.repec.org/n?u=RePEc:cdl:econwp:qt3883h8j1&r=sog |
| By: | Adler, Susanne Jana (Ludwig-Maximilians-University Munich); Sharma, Pratyush N (The University of Alabama); Radomir, Lăcrămioara (Faculty of Economics and Business Administration, Babeș-Bolyai University) |
| Abstract: | Driven by the high-profile failures to reproduce and replicate published findings, there have been increasing demands to adopt open science practices across scientific disciplines in order to enhance research transparency. Critics have highlighted the use of underpowered studies and researchers’ analytical degrees of freedom as factors contributing to these issues. Despite methodological advances and updated guidelines, similar concerns persist regarding studies utilizing partial least squares structural equation modeling (PLS-SEM). Open science practices can help alleviate these concerns by facilitating transparency in PLS-SEM-based studies. However, the current level of adherence to these practices remains unknown. In this article, we conduct a comprehensive literature review of leading marketing journals to assess the extent to which open science practices are implemented in PLS-SEM-based studies. Based on the observed lack of adoption, we propose a PLS-SEM-specific preregistration template that researchers can use to foster transparency in their analyses, thereby bolstering confidence in their findings. |
| Date: | 2023–09–23 |
| URL: | http://d.repec.org/n?u=RePEc:osf:osfxxx:sbpe9&r=sog |
| By: | Ramani, Ravi S.; Aguinis, Herman; Coyle-Shapiro, Jacqueline A.M. |
| Abstract: | We address the grossly incorrect inferences that result from using journal impact factor (JIF) as a proxy to assess individual researcher and article scholarly impact. This invalid practice occurs because of confusion about the definition and measurement of impact at different levels of analysis. Specifically, JIF is a journal-level measure of impact, computed by aggregating citations of individual articles (i.e., upward effect), and is therefore inappropriate when measuring impact at lower levels of analysis, such as that of individual researchers, or of individual articles published in a particular journal (i.e., downward effect). We illustrate the severity of the errors that occur when using JIF to evaluate individual scholarly impact, and advocate for an immediate moratorium on the exclusive use of JIF and other journal-level (i.e., higher level of analysis) measures when assessing the impact of individual researchers and individual articles (i.e., lower level of analysis). Given the importance and interest in assessing the scholarly impact of researchers and articles, we delineate level-appropriate and readily available measures. |
| JEL: | J50 |
| Date: | 2022–09–21 |
| URL: | http://d.repec.org/n?u=RePEc:ehl:lserod:117286&r=sog |
| By: | Committee, Nobel Prize (Nobel Prize Committee) |
| Abstract: | Women are severely underrepresented in the global labor market: around 50% of women work or actively seek work for income, compared to 80% for men. The gender differences in participation are fundamentally driven by variation in women’s participation rates – men’s participation rates are broadly constant across time and countries. The participation gaps between men and women are particularly large in South Asia, the Middle East, and North Africa, where they sometimes exceed 50 percentage points. |
| Keywords: | Gender in labor markets; |
| JEL: | J70 J71 J78 |
| Date: | 2023–10–09 |
| URL: | http://d.repec.org/n?u=RePEc:ris:nobelp:2023_002&r=sog |