Abstract: |
The paper focuses on the robustness of rankings of academic journal quality
and research impact in general, and in Economics, in particular, based on the
widely-used Thomson Reuters ISI Web of Science citations database (ISI). The
paper analyses 299 leading international journals in Economics using
quantifiable Research Assessment Measures (RAMs), and highlights the
similarities and differences in various RAMs, which are based on alternative
transformations of citations. All existing RAMs to date have been static, so
two new dynamic RAMs are developed to capture changes in impact factor over
time and escalating journal self citations. Alternative RAMs may be calculated
annually or updated daily to determine When, Where and How (frequently)
published papers are cited (see Chang et al. (2011a, b, c)). The RAMs are
grouped in four distinct classes that include impact factor, mean citations
and non-citations, journal policy, number of high quality papers, and journal
influence and article influence. These classes include the most widely used
RAMs, namely the classic 2-year impact factor including journal self citations
(2YIF), 2-year impact factor excluding journal self citations (2YIF*), 5-year
impact factor including journal self citations (5YIF), Eigenfactor (or Journal
Influence), Article Influence, h-index, and PI-BETA (Papers Ignored - By Even
The Authors). As all existing RAMs to date have been static, two new dynamic
RAMs are developed to capture changes in impact factor over time (5YD2 =
5YIF/2YIF) and Escalating Self Citations. We highlight robust rankings based
on the harmonic mean of the ranks of RAMs across the 4 classes. It is shown
that emphasizing the 2-year impact factor of a journal, which partly answers
the question as to When published papers are cited, to the exclusion of other
informative RAMs, which answer Where and How (frequently) published papers are
cited, can lead to a distorted evaluation of journal quality, impact and
influence relative to the harmonic mean of the ranks. |
Keywords: |
Research assessment measures, Impact factor, IFI, C3PO, PI-BETA, STAR, Eigenfactor, Article Influence, h-index, 5YD2, ESC, harmonic mean of the ranks, economics, journal rankings. |