Instance-based learning

In machine learning, instance-based learning (sometimes called memory-based learning[1]) is a family of learning algorithms that, instead of performing explicit generalization, compares new problem instances with instances seen in training, which have been stored in memory. Instance-based learning is a kind of lazy learning.

It is called instance-based because it constructs hypotheses directly from the training instances themselves.[2] This means that the hypothesis complexity can grow with the data:[2] in the worst case, a hypothesis is a list of n training items and the computational complexity of classifying a single new instance is O(n). One advantage that instance-based learning has over other methods of machine learning is its ability to adapt its model to previously unseen data: instance-based learners may simply store a new instance or throw an old instance away.

Examples of instance-based learning algorithm are the k-nearest neighbor algorithm, kernel machines and RBF networks.[3]:ch. 8 These store (a subset of) their training set; when predicting a value/class for a new instance, they compute distances or similarities between this instance and the training instances to make a decision.

To battle the memory complexity of storing all training instances, as well as the risk of overfitting to noise in the training set, instance reduction algorithms have been proposed.[4]

Gagliardi[5] applies this family of classifiers in medical field as second-opinion diagnostic tools and as tools for the knowledge extraction phase in the process of knowledge discovery in databases. One of these classifiers (called Prototype exemplar learning classifier (PEL-C) is able to extract a mixture of abstracted prototypical cases (that are syndromes) and selected atypical clinical cases.

See also

References

  1. ↑ Walter Daelemans; Antal van den Bosch (2005). Memory-Based Language Processing. Cambridge University Press.
  2. 1 2 Stuart Russell and Peter Norvig (2003). Artificial Intelligence: A Modern Approach, second edition, p. 733. Prentice Hall. ISBN 0-13-080302-2
  3. ↑ Tom Mitchell (1997). Machine Learning. McGraw-Hill.
  4. ↑ D. Randall Wilson; Tony R. Martinez (2000). "Reduction techniques for instance-based learning algorithms". Machine Learning (Kluwer).
  5. ↑ Gagliardi, F (2011). "Instance-based classifiers applied to medical databases: Diagnosis and knowledge extraction". Artificial Intelligence in Medicine 52 (3): 123–139. doi:10.1016/j.artmed.2011.04.002.
This article is issued from Wikipedia - version of the Thursday, January 28, 2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.