Bayes error rate

In statistical classification, the Bayes error rate is the lowest possible error rate for any classifier of a random outcome (into, for example, one of two categories) and is analogous to the irreducible error.[1][2]

A number of approaches to the estimation of the Bayes error rate exist. One method seeks to obtain analytical bounds which are inherently dependent on distribution parameters, and hence difficult to estimate. Another approach focuses on class densities, while yet another method combines and compares various classifiers.[2]

The Bayes error rate finds important use in the study of patterns and machine learning techniques.[3]

Error determination

In terms of machine learning and pattern classification, the labels of a set of random observations can be divided into 2 or more classes. Each observation is called an instance and the class it belongs to is the label. The Bayes error rate of the data distribution is the probability an instance is misclassified by a classifier that knows the true class probabilities given the predictors. For a multiclass classifier, the Bayes error rate may be calculated as follows:

p = \sum_{C_{i} \neq C_\text{max}} \textstyle \int\limits_{x\in H_{i}}P(x|C_{i})p(C_{i})\, dx,

where x is an instance, Ci is a class into which an instance is classified, Hi is the area/region that a classifier function h classifies as Ci.

The Bayes error is non-zero if the classification labels are not deterministic, i.e., there is a non-zero probability of a given instance belonging to more than one class.

See also

References

  1. ↑ Fukunaga, Keinosuke (1990) Introduction to Statistical Pattern Recognition by ISBN 0122698517 pages 3 and 97
  2. 1 2 K. Tumer, K. (1996) "Estimating the Bayes error rate through classifier combining" in Proceedings of the 13th International Conference on Pattern Recognition, Volume 2, 695–699
  3. ↑ Hastie, Trevor. The Elements of Statistical Learning (2nd ed.). http://statweb.stanford.edu/~tibs/ElemStatLearn/: Springer. p. 17. ISBN 978-0387848570.


This article is issued from Wikipedia - version of the Thursday, December 03, 2015. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.