Incremental Machine Learning System

An Incremental Machine Learning System is a Machine Learning System, based on Incremental Learning, that implements an Incremental Machine Learning Algorithm to solve a Incremental Machine Learning Task.

• AKA: Incremental Learning System.
• Context:
  • It ranges from being an Incremental Unsupervised Learning System to being an Incremental Supervised Learning System.
• Example(s)
  • An Active Learning System,
  • A Cumulative Learning System,
  • A Decision Tree System such as:
    • an ID4,
    • an ID5R,
  • An artificial neural network such as:
    • a Fuzzy ARTMAP,
    • an IGNG,
    • a Learn++,
    • a TopoART,
  • an Incremental SVM,
  • An Online Learning System.
  • …
• Counter-Example(s):
  • A Batch Learning System,
  • A Offline Learning System.
• See: Active Learning; Decision Tree, Decision Rules, Artificial Neural Network, Support Vector Machine, Forecasting, Machine Learning, Supervised Learning, Unsupervised Learning, Data Stream, Big Data, Classification.

---

---

References

2018

• (Wikipedia, 2018) ⇒ https://en.wikipedia.org/wiki/Incremental_learning Retrieved:2018-4-15.
  • In computer science, incremental learning is a method of machine learning, in which input data is continuously used to extend the existing model's knowledge i.e. to further train the model. It represents a dynamic technique of supervised learning and unsupervised learning that can be applied when training data becomes available gradually over time or its size is out of system memory limits. Algorithms that can facilitate incremental learning are known as incremental machine learning algorithms.

    Many traditional machine learning algorithms inherently support incremental learning, other algorithms can be adapted to facilitate this. Examples of incremental algorithms include decisions trees (IDE4, ID5R ), decision rules, artificial neural networks (RBF networks, Learn++, Fuzzy ARTMAP, TopoART, and IGNG ) or the incremental SVM. The aim of incremental learning is for the learning model to adapt to new data without forgetting its existing knowledge, it does not retrain the model. Some incremental learners have built-in some parameter or assumption that controls the relevancy of old data, while others, called stable incremental machine learning algorithms, learn representations of the training data that are not even partially forgotten over time. Fuzzy ART and TopoART are two examples for this second approach.

    Incremental algorithms are frequently applied to data streams or big data, addressing issues in data availability and resource scarcity respectively. Stock trend prediction and user profiling are some examples of data streams where new data becomes continuously available. Applying incremental learning to big data aims to produce faster classification or forecasting times.

2017

• (Utgoff, 2017) ⇒ Paul E. Utgoff. (2017). Incremental Learning. In: Sammut, C., Webb, G.I. (eds) Encyclopedia of Machine Learning and Data Mining. Springer, Boston, MA
  • QUOTE: Incremental learning refers to any online learning process that learns the same model as would be learned by a batch learning algorithm.

    (...)

     Incremental learning is useful when the input to a learning process occurs as a stream of distinct observations spread out over time, with the need or desire to be able to use the result of learning at any point in time, based on the input observations received so far. In principle, the stream of observations may be infinitely long, or the next observation long delayed, precluding any hope of waiting until all the observations have been received. Without the ability to forestall learning, one must commit to a sequence of hypotheses or other learned artifacts based on the inputs observed up to the present. One would rather not simply accumulate and store all the inputs and, upon receipt of each new one, apply a batch learning algorithm to the entire sequence of inputs received so far. It would be preferable computationally if the existing hypothesis or other artifact of learning could be updated in response to each newly received input observation.

2012

• (Seel,2012) ⇒ (2012) "Incremental Learning". In: Seel N.M. (eds) Encyclopedia of the Sciences of Learning. Springer, Boston, MA
  • QUOTE: The term “incremental learning” is often used for sequential or constructive learning in contrast to batch or epoch learning (Bertsekas and Tsitsiklis 1996). Incremental learning is based on the principle of starting with simple and basic principles before advancing to more complex information. Incremental learning happens in bits and pieces, and successful retention of knowledge is based upon previously attained knowledge. As a style of acquiring knowledge and skills, the concept of incremental learning can be found in psychology as well as in machine learning and refers to situations where input data come only in sequence, and a timely updating model is crucial for actions. In psychology, the term “incremental learning” can be traced back to Thorndike but can also be found in more recent theories of how people learn (e.g., Bransford et al. 2000). However, the term “incremental learning” plays an important role also in the field of machine learning which includes algorithms for …

2005

• (Ferrer-Troyano et al., 2005) ⇒ Ferrer-Troyano, F., Aguilar-Ruiz, J. S., & Riquelme, J. C. (2005, March). "Incremental rule learning based on example nearness from numerical data streams" (PDF). In: Proceedings of the 2005 ACM symposium on Applied computing (pp. 568-572). ACM.
  • ABSTRACT: Mining data streams is a challenging task that requires online systems based on incremental learning approaches. This paper describes a classification system based on decision rules that may store up–to–date border examples to avoid unnecessary revisions when virtual drifts are present in data. Consistent rules classify new test examples by covering and inconsistent rules classify them by distance as the nearest neighbor algorithm. In addition, the system provides an implicit forgetting heuristic so that positive and negative examples are removed from a rule when they are not near one another.

---