Learning Algorithm

A Learning Algorithm is an algorithm that can be implemented into a learning system to solve a learning task (to improve task performance on some learning task over multiple algorithm runs).

• AKA: Induction Algorithm, Learning Process, Inducer, Data-Driven Algorithm, Data-Centric Algorithm.
• Context:
  • Input:
    • a Data Record Set (a set of Examples)
  • Output:
    • A Predictive Model
  • It can range from being an Unsupervised Learning Algorithm to being a Supervised Learning Algorithm.
  • It can range from being a Model-based Learning Algorithm to being an Instance-based Learning Algorithm
  • It can be:
    • an Eager Learning Algorithm, which produces a portable Predictive Model that can independently handle all future Testing Records.
    • a Lazy Learning Algorithm, whose predictions are always calculated at the time of the request.
  • It can be:
    • a Feature-based Learning Algorithm.
    • a Kernel-based Learning Algorithm, that maps the record into a Kernel Space.
  • It can be:
    • a Tuple-based Learning Algorithm, where the records are unrelated Tuples.
    • a Relational Learning Algorithm, where the relations between records can play a role in the Predictive Model.
  • It can be:
    • a Symbolic Learning Algorithm, where the Predictive Model is based on a Symbolic Representation.
    • a Black-Box Learning Algorithm, where the Predictive Model is not inspectable.
  • It can be an Ensemble Learning Algorithm.
  • It can range from:
    • being an Online Learning Algorithm,
    • to being a Batch Learning Algorithm.
  • It can be implemented in a Learning System.
• Example(s):
  • a Machine Learning Algorithm.
    • a Reinforcement Learning Algorithm.
    • a Inductive Logic Algorithm.
  • a Statistical Algorithm.
    • Support Vector Machine Algorithm.
    • k-Nearest Neighbor Learning Algorithm.
    • Naive-Bayes Learning Algorithm.
    • Neural Network Learning Algorithm.
    • Linear Least-Squares Fit Algorithm.
  • a Data Mining Algorithm.
    • a Decision Tree Algorithm.
• Counter-Example(s):
  • a Deductive Algorithm.
  • an Abductive Algorithm.
• See: Inductive Reasoning, Function Approximation Algorithm, Data Mining Algorithm.

---

---

References

2009

• http://en.wikipedia.org/wiki/Learning
  • Learning is acquiring new knowledge, behaviors, skills, values, preferences or understanding, and may involve synthesizing different types of information. The ability to learn is possessed by humans, animals and some machines. Progress over time tends to follow learning curves.
  • Human learning may occur as part of education or personal development. It may be goal-oriented and may be aided by motivation. The study of how learning occurs is part of neuropsychology, educational psychology, learning theory, and pedagogy.
  • Learning may occur as a result of habituation or classical conditioning, seen in many animal species, or as a result of more complex activities such as play, seen only in relatively intelligent animals

2006

• (Mitchell, 2006) ⇒ Tom M. Mitchell. (2006). "The Discipline of Machine Learning." Machine Learning Department technical report CMU-ML-06-108, Carnegie Mellon University.
  • A scientific field is best defined by the central question it studies. The field of Machine Learning seeks to answer the question “How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?”

1998

• (Kohavi & Provost, 1998) ⇒ Ron Kohavi, and Foster Provost. (1998). "Glossary of Terms." In: Machine Leanring 30(2-3).
  • Inducer / induction algorithm: An algorithm that takes as input specific instances and produces a model that generalizes beyond these instances.

---

---