nep-sog New Economics Papers
on Sociology of Economics
Issue of 2022‒03‒07
six papers chosen by
Jonas Holmström
Axventure AB

  1. Do Negative Replications Affect Citations? By Tom Coupé; W. Robert Reed
  2. Sex-based sorting among economists: Evidence from the NBER By Davies, Benjamin
  3. Research Excellence or Scientific Advisory Boards – Investigating German and US Agri-Environmental Boards By Göser, Maya; Wimmer, Stefan; Sauer, Johannes
  4. Publishing in the social sciences and its representation in research evaluation and funding systems By Sivertsen, Gunnar
  5. Research equity: Incentivizing high-risk basic research with market mechanisms By Ellgen, Clifford; Kang, Dominique
  6. The use of bibliometrics in assessments of social scientists By Sivertsen, Gunnar

  1. By: Tom Coupé (University of Canterbury); W. Robert Reed (University of Canterbury)
    Abstract: This study examines the effect of negative replications on the citation rates of replicated studies. We study a set of 204 replicated studies in economics and compare their citation performance with an initial sample of 112,000 potential controls taken from Scopus. From this initial pool, we match each replicated study with multiple controls based on having comparable citation histories. We have two main findings. First, studies that are replicated receive somewhat more citations than their matched control studies. Second, there is no evidence that studies that receive negative replications suffer a penalty in the form of fewer citations.
    Keywords: Replications, Citations, Matching, Meta-science, Self-correcting science
    JEL: A11 A14 B41 C18
    Date: 2022–02–01
  2. By: Davies, Benjamin
    Abstract: I compare the co-authorship patterns of male and female economists using historical data on National Bureau of Economic Research working papers. Males tended to work in smaller teams than females, but co-authored more papers and so had more co-authors overall. Males and females co-authored with members of the same sex more often than would be expected if co-authorships were random. This was especially true for males in Macro/Finance.
    Date: 2022–01–20
  3. By: Göser, Maya; Wimmer, Stefan; Sauer, Johannes
    Abstract: Scientific advisory boards are frequently established to provide scientific insights and advice to policy makers. Advisory board appointing bodies often state that research excellence and scientific seniority are the main grounds on which advisory board members are selected. Many authors have pointed out that there is more to giving good scientific advice than just being an expert for a specific research field. The aim of this study is to analyse if and how research excellence correlates with the probability of being appointed as a scientific advisory board member. We have collected data for scientific advisory boards from both the US and Germany. Our dataset includes current board members and a control group of non-board-members with similar expertise. We use logit regression models to analyse how research excellence correlates with the probability of appointment to a scientific advisory board. We use the h-index as a highly cited proxy for individual research excellence and factor in the research excellence status of the associated institution as well as other potentially correlating factors indicating scientific seniority. Our results suggest that research excellence is insignificant or even correlates negatively with the probability of being appointed to a scientific advisory board.
    Keywords: Agricultural and Food Policy, Political Economy
    Date: 2021–11–18
  4. By: Sivertsen, Gunnar
    Abstract: The paper describes the full range of publishing and its purposes in the social sciences – from scholarly publishing via professional communication to societal interaction in public media – and how it is represented in five different contexts for research evaluation and funding. The five contexts are: applications for external project funding, applications for positions or promotions, indicator-based institutional funding systems, summative organizational evaluation systems, and formative organizational evaluation systems. The chapter provides a critical discussion of how publications from the social sciences may be filtered out or placed in predetermined hierarchies in these evaluation and funding contexts, and also of how the evaluation and funding procedures can be improved to appropriately represent social science research and publishing.
    Date: 2022–01–03
  5. By: Ellgen, Clifford; Kang, Dominique
    Abstract: Innovation in basic research is vital to scientific progress and technological development; however, such research finds insufficient support in the current research environment. To stimulate high-risk, high-reward basic research, this paper proposes a “research equity” funding model in which funders—such as government agencies and philanthropies—would pay researchers and institutions for completed research: The more valuable the research, the greater the reward. The valuation of completed research could be done with a novel “chess rating” method: A peer reviewer would be presented with a pair of research papers and would decide which of the two is of greater value, and a large number of comparisons would produce a numerical rating to inform payment. Payment based on research value would enable many of the qualities found in healthy markets. Initial capital for basic research would be provided by research institutions, which would be financially incentivized to invest in a diverse body of basic research that includes both low-risk, conservative research and high-risk, innovative research. Institutions would be motivated to demonstrate the value of completed research in their portfolios, which may accelerate recognition of important results. By motivating researchers and institutions to produce and promote valuable research, the research equity model could stimulate more rapid scientific discovery and progress. Notably, the research equity model could coexist with grant funding.
    Date: 2021–10–21
  6. By: Sivertsen, Gunnar
    Abstract: The paper is focused on practical advice for the use of bibliometrics in research assessment in the social sciences. Guidelines are presented from three official sources of advice with a particular focus on individual-level assessments of applications for positions, promotions, and external funding. General problems with applying bibliometrics in evaluations of the social sciences are also discussed, as well as the specific problems with using the Journal Impact Factor and the H-Index. The conclusion is not that bibliometrics should be avoided in research assessment of social scientists. Used with care and competence, bibliometrics can be a valuable extra source of information, but not replace judgement in research evaluation.
    Date: 2022–01–03

This nep-sog issue is ©2022 by Jonas Holmström. It is provided as is without any express or implied warranty. It may be freely redistributed in whole or in part for any purpose. If distributed in part, please include this notice.
General information on the NEP project can be found at For comments please write to the director of NEP, Marco Novarese at <>. Put “NEP” in the subject, otherwise your mail may be rejected.
NEP’s infrastructure is sponsored by the School of Economics and Finance of Massey University in New Zealand.