nep-sog New Economics Papers
on Sociology of Economics
Issue of 2008‒10‒07
two papers chosen by
Jonas Holmström
Swedish School of Economics and Business Administration

  1. Do Rankings Reflect Research Quality? By Bruno S. Frey; Katja Rost
  2. Differences in Impact Factor Across Fields and Over Time By Benjamin Althouse; Jevin West; Ted Bergstrom; Carl Bergstrom

  1. By: Bruno S. Frey; Katja Rost
    Abstract: Publication and citation rankings have become major indicators of the scientific worth of universities and countries, and determine to a large extent the career of individual scholars. We argue that such rankings do not effectively measure research quality, which should be the essence of evaluation. For that reason, an alternative ranking is developed as a quality indicator, based on membership on academic editorial boards of professional journals. It turns out that especially the ranking of individual scholars is far from objective. The results differ markedly, depending on whether research quantity or research quality is considered. Even quantity rankings are not objective; two citation rankings, based on different samples, produce entirely different results. It follows that any career decisions based on rankings are dominated by chance and do not reflect research quality. Instead of propagating a ranking based on board membership as the gold standard, we suggest that committees make use of this quality indicator to find members who, in turn, evaluate the research quality of individual scholars.
    Keywords: Rankings; Universities; Scholars; Publications; Citations
    JEL: H43 L15 O38
    Date: 2008–09
  2. By: Benjamin Althouse (University of Washington); Jevin West (University of Washington); Ted Bergstrom (University of California, Santa Barbara); Carl Bergstrom (University of Washington)
    Abstract: The impact factor of an academic journal for any year is the number of times the average article published in that journal in the previous two years are cited in that year. From 1994-2005, the average impact factor of journals listed by the ISI has been increasing by an average of 2.6 percent per year. This paper documents this growth and explores its causes.
    Keywords: academic journals, citations, impact factor,
    Date: 2008–04–23

This nep-sog issue is ©2008 by Jonas Holmström. It is provided as is without any express or implied warranty. It may be freely redistributed in whole or in part for any purpose. If distributed in part, please include this notice.
General information on the NEP project can be found at For comments please write to the director of NEP, Marco Novarese at <>. Put “NEP” in the subject, otherwise your mail may be rejected.
NEP’s infrastructure is sponsored by the School of Economics and Finance of Massey University in New Zealand.