| Abstract: |
The area under Receiver Operating Characteristic (ROC) curve, also known as
the AUC-index, is commonly used for ranking the performance of data mining
models. The AUC has many merits, such as objectivity and ease of
interpretation. However, since it is class indifferent, its usefulness while
dealing with highly skewed data sets is questionable, to say the least. In
this paper, we propose a simple alternative scalar measure to the AUCindex,
the Area Under an Kappa curve (AUK). The proposed AUK-index compensates for
the above basic flaw of the AUC by being sensitive to the class distribution.
Therefore it is particularly suitable for measuring classifiers’ performance
on skewed data sets. After introducing the AUK we explore its mathematical
relationship with the AUC and show that there is a nonlinear relation between
them. |