Entropy-Minimization Algorithm

An Entropy-Minimization Algorithm is an optimization algorithm that minimizes an entropy function.

• Context:
  • It can be applied by an Entropy-Minimization System (that solves an entropy-minimization task).
• See: Maximum Entropy Algorithm, Shannon Entropy, EM Algorithm.

---

---

References

2005

• (Grandvalet & Bengio, 2004) ⇒ Yves Grandvalet, and Yoshua Bengio. (2004). “Semi-supervised Learning by Entropy Minimization.” In: Advances in Neural Information Processing Systems, pp. 529-536.

2003

• (Zhu, Ghahramani & Lafferty, 2003) ⇒ Xiaojin Zhu, Zoubin Ghahramani, and John D. Lafferty. (2003). “Semi-supervised learning using Gaussian fields and harmonic functions.” In: Proceedings of the 20th International Conference on Machine Learning (ICML 2003).
  • QUOTE: We also propose a method of parameter learning by entropy minimization, and show the algorithm’s ability to perform feature selection.

2002

• (Erdogmus & Principe, 2002) ⇒ Deniz Erdogmus, and Jose C. Principe. (2002). “An Error-entropy Minimization Algorithm for Supervised Training of Nonlinear Adaptive Systems." Signal Processing, IEEE Transactions on 50, no. 7

---