Baum-Welch Algorithm

A Baum-Welch Algorithm is an Expectation-Maximization (EM) algorithm that can fit a Hidden Markov Model.

• See: Forward-Backward Algorithm.

---

---

References

2017

• (Sammut & Webb, 2017) ⇒ (2017) Baum-Welch Algorithm. In: Sammut, C., Webb, G.I. (eds) Encyclopedia of Machine Learning and Data Mining. Springer, Boston, MA
  • QUOTE: The Baum–Welch algorithm is used for computing maximum likelihood estimates and posterior mode estimates for the parameters (transition and emission probabilities) of a HMM, when given only output sequences (emissions) as training data.

    The Baum–Welch algorithm is a particular instantiation of the expectation-maximization algorithm, suited for HMMs.

2015a

• (Wikipedia, 2015) ⇒ http://en.wikipedia.org/wiki/Baum–Welch_algorithm Retrieved:2015-11-11.
  • In electrical engineering, computer science, statistical computing and bioinformatics, the Baum–Welch algorithm is used to find the unknown parameters of a hidden Markov model (HMM). It makes use of the forward-backward algorithm and is named for Leonard E. Baum and Lloyd R. Welch.

2015b

• (Tu, 2015) & Stephen Tu (2015). "Derivation of Baum-Welch Algorithm for Hidden Markov Models".
  • QUOTE: Baum-Welch is an iterative procedure for estimating [math]\displaystyle{ \theta^* }[/math] from only [math]\displaystyle{ \mathcal{X} }[/math] . It works by maximizing a proxy to the log-likelihood, and updating the current model to be closer to the optimal model. Each iteration of Baum-Welch is guaranteed to increase the log-likelihood of the data. But of course, convergence to the optimal solution is not guaranteed.

     Baum-Welch can be described simply as repeating the following steps until convergence:

1. Compute [math]\displaystyle{ Q(\theta, \theta_s ) = \sum_{z\in Z} \log [P(X , z; \theta)] P(z|X ; \theta_s ) }[/math].

2. Set [math]\displaystyle{ \theta^{s+1} = \underset{\theta}{\operatorname{arg\,max}} \,Q(\theta, \theta_s ) }[/math]

---