nep-sog New Economics Papers
on Sociology of Economics
Issue of 2018‒03‒12
two papers chosen by
Jonas Holmström
Axventure AB

  1. Publication Performance vs. Influence: On the Questionable Value of Quality Weighted Publication Rankings By Justus Haucap; Tobias Thomas; Klaus Wohlrabe
  2. Rising Stars By Erich Battistin; Marco Ovidi

  1. By: Justus Haucap; Tobias Thomas; Klaus Wohlrabe
    Abstract: In broad parts of the scientific community the position in publication performance rankings, based on journal quality ratings is seen as highly reputational for the scientist. This contribution provides evidence that, at least in economics, such publication performance measures can not always be reconciled with measures for academic influence such as citation-based measures. We analyze data from the Scopus database as well as from the prestigious German-based Handelsblatt ranking for 100 renowned economists (lifetime achievement). Scholarly influence is proxied by various bibliometric indicators such as the number of citations, the h-index, the citations of the most cited paper as well as the hardly honorable Pi-Beta-score (“Publications Ignored, By Even The Author(s)”). We argue that publication performance measures based on journal ratings, such as the Handelsblatt rankings, are not good proxies for an economist’s impact within the scientific community. From this perspective the value of publication performance rankings based on journal quality ratings is questionable.
    Keywords: economics, academic reputation, academic rankings, influence, citations, Scopus, Handelsblatt ranking, academic journals
    JEL: A12 A14
    Date: 2017
  2. By: Erich Battistin (Queen Mary University of London); Marco Ovidi (Queen Mary University of London)
    Abstract: We use the UK’s 2014 Research Excellence Framework (REF) to study which attributes characterize a top-scoring (four-star) publication in Economics and Econometrics. We frame the analysis as a classification problem and, using information in official documents, derive conditions to infer the unobservable score that panellists awarded to each publication. Juxtaposing institutions’ submissions with REF outcomes provides information on the latent pass-marks used for assigning quality levels, which respond to journal prestige measured by the Thomson Reuters Article Influence Score. We explore this statistical feature in the econometric analysis, which reveals the limited contribution to awarded quality made by other publication attributes, possibly unobservable to us, conditional on the Article Influence Score. We conclude that, in large-scale and costly evaluations such as the REF, the time-consuming task of peer reviews should be devoted to publications not in academic outlets with unambiguously top-scoring bibliometric indicators of journal impact. Our model also predicts a ranking of academic journals consistent with the classification of REF panellists.
    Keywords: Education Policy, Higher Education, Journal Ranking, Research Funding
    JEL: H52 H83 I23 I28
    Date: 2017–12–12

This nep-sog issue is ©2018 by Jonas Holmström. It is provided as is without any express or implied warranty. It may be freely redistributed in whole or in part for any purpose. If distributed in part, please include this notice.
General information on the NEP project can be found at For comments please write to the director of NEP, Marco Novarese at <>. Put “NEP” in the subject, otherwise your mail may be rejected.
NEP’s infrastructure is sponsored by the School of Economics and Finance of Massey University in New Zealand.