The Markov model is a stochastic model in which the state of the random variable at the next instance of time depends only on the outcome of the random variable at the current time. The simplest kind of Markov model is a Markov chain, which we discussed in Chapter 1, Introduction to Markov Process.

Suppose we have a set of sequential observations (x₁,. . ., x_n) obeying the Markov property, then we can state the joint probability distribution for N observations as the following:

Graphical representation of a first-order Markov chain in which the distribution of the current observation is conditioned on the value of the previous observation

The preceding representation of the Markov chain is different from the representations we saw earlier. In this representation, the observations are presented as nodes and the edges represent conditional probability between...

An HMM is a specific case of state space model in which the latent variables are discrete and multinomial variables. From the graphical representation, we can also consider an HMM to be a double stochastic process consisting of a hidden stochastic Markov process (of latent variables) that we cannot observe directly, and another stochastic process that produces a sequence of the observation given the first process.

Before moving on to the parameterization, let's consider an example of coin-tossing to get an idea of how it works. Assume that we have two unfair coins, M₁ and M₂, with M₁ having a higher probability (70%) of getting heads and M₂ having a higher probability (80%) of getting tails. Someone sequentially flips these two coins, however, we do not know which one. We can only observe the outcome, which can either be heads (H) or tails (T):

We can consider the...

In the previous section, we discussed generating an observation sequence of a given HMM. But, in reality, most of the time we are not interested in generating the observation sequence, mostly because we don't know the parameters of the HMM to generate observations in the first place.

For a given HMM representation, in most of the applications, we are always trying to address the following three problems:

• Evaluation of the model: Given the parameters of the model and the observation sequence, estimating the probability of the sequence
• Predicting the optimal sequence: Given the parameters of the model and the observation sequence, estimating the most probable sequence of the state sequence that had produced these observations
• Parameter-learning: Given a sequence of observations, estimating the parameters of the HMM model that generated it

In this section...

In the previous sections, we discussed HMM, sampling from it and evaluating the probability of a given sequence given its parameters. In this section, we are going to discuss some of its variations.

Let's consider the problem of modelling of several objects in a sequence of images. If there are M objects with K different positions and orientations in the image, there are be K^M possible states for the system underlying an image. An HMM would require K^M distinct states to model the system. This way of representing the system is not only inefficient but also difficult to interpret. We would prefer that our HMM could capture the state space by using M different K-dimensional variables.

A factorial...

In this chapter, we got a detailed introduction to Markov model and HMM. We talked about parameterizing an HMM, generating samples from it, and their code. We discussed estimating the probability of observation, which would form the basis of inference, which we'll cover in the next chapter. We also talked about various extensions of HMMs.

In the next chapter, we will take an in-depth look at inference in HMMs.