| Abstract: |
Publication and citation rankings have become major indicators of the
scientific worth of universities and countries, and determine to a large
extent the career of individual scholars. We argue that such rankings do not
effectively measure research quality, which should be the essence of
evaluation. For that reason, an alternative ranking is developed as a quality
indicator, based on membership on academic editorial boards of professional
journals. It turns out that especially the ranking of individual scholars is
far from objective. The results differ markedly, depending on whether research
quantity or research quality is considered. Even quantity rankings are not
objective; two citation rankings, based on different samples, produce entirely
different results. It follows that any career decisions based on rankings are
dominated by chance and do not reflect research quality. Instead of
propagating a ranking based on board membership as the gold standard, we
suggest that committees make use of this quality indicator to find members
who, in turn, evaluate the research quality of individual scholars. |