Semi-Markov Chain

A Semi-Markov Chain is a Hidden Markov Model that allows each State s_i to persist for a non-unit length of time d_i.

• See: Semi-Markov CRF, Hidden Semi-Markov Model, Semi-Markov Process.

---

---

References

2009

• http://en.wikipedia.org/wiki/Semi-Markov_process

2008

• (Stefan Barbu & Limnios, 2008) ⇒ Vlad Stefan Barbu, and Nikolaos Limnios. (2008). “Semi-Markov Chains and Hidden Semi-Markov Models toward Applications.” In: Lecture Notes in Statistics, Vol. 191. Springer. ISBN:978-0-387-73171-1
  • This book is concerned with the estimation of discrete-time semi-Markov and hidden semi-Markov processes. Semi-Markov processes are much more general and better adapted to applications than the Markov ones because sojourn times in any state can be arbitrarily distributed, as opposed to the geometrically distributed sojourn time in the Markov case. Another unique feature of the book is the use of discrete time, especially useful in some specific applications where the time scale is intrinsically discrete. The models presented in the book are specifically adapted to reliability studies and DNA analysis.
• (Lester Lipsky, 2008) ⇒ Lester Lipsky. (2008). “Semi Markov Process.” In: Queueing Theory. Springer. doi:10.1007/978-0-387-49706-8_8 ISBN:978-0-387-49706-8
  • In many (if not most) real-world applications, the arrival of customers to a service center is not well described by renewal processses. Quite often, the times between successive arrivals are correlated, whereas renewal processes have independent interarrival times. A natural generalization is the class of semi-Markov processes (SMP), which when specifically applied to the arrival of customers are called Markov Renewal Processes (MRP) or Markov Arrival Processes (MAP). Of course arrivals to one station correspond to departures from some other station. So, to avoid confusion, we use the terms SMP or MRP here.

2004

• (Sarawagi & Cohen, 2004) ⇒ Sunita Sarawagi, and William W. Cohen. (2004). “Semi-Markov Conditional Random Fields for Information Extraction.” In: Proceedings of Advances in Neural Information Processing Systems, 17 NIPS 2004.
  • We describe semi-Markov conditional random-fields (semi-CRFs), a conditionally trained version of semi-Markov chains. ... Recall that semi-Markov chain models extend hidden Markov models (HMMs) by allowing each state s_i to persist for a non-unit length of time d_i. After this time has elapsed, the system will transition to a new state s₀, which depends only on s_i ; however, during the “segment” of time between [math]\displaystyle{ i }[/math] to [math]\displaystyle{ i }[/math] + d_i, the behavior of the system may be non-Markovian. Semi-Markov models are fairly common in certain applications of statistics [8, 9], and are also used in reinforcement learning to model hierarchical Markov decision processes [19].

---