Markov Models
A Markov chain is a probabilistic model describing a sequence of possible events that satisfies the Markov property.
Markov property: In a sequence or stochastic process that possesses the Markov property, the probability of each event depends only on the immediately preceding state (rather than earlier states). These sequences or processes can also be called Markovian, or a Markov Process.
Named after Russian mathematician Andrey Markov, the Markov property is very desirable since it significantly reduces the complexity of a problem. In forecasting, instead of taking into account all previous states, t-1, t-2, …, 0, only t-1 is considered.
Similarly, the Markov assumption, for a mathematical or machine learning model is that the sequence satisfies the Markov property. In models such as the Markov chain and Hidden Markov model, the process or sequence is assumed to be a Markov process.
In a discrete-time Markov chain (DTMC), the...
