Generating Text with Markov Chains

One of the more interesting uses of NLP is generating text, whether for amusement, for research, or for profit. The key concept in generating text is to take advantage of the statistical properties of the text. This makes the output text more realistic and, more importantly, we gain information about the text itself if we can rearrange it to identify interesting patterns in it. Text can be generated in many different ways. We will explore doing so using Markov chains.

Markov Chains

A Markov chain is a mathematical system for transitioning between states based on probabilities. No matter how a system arrives at its present state, the possible future states are all predetermined. Furthermore, the move to the next state is based on a set of probabilities. Markov chains operate in timesteps. After each timestep, the system selects a new state. This new state is chosen based on the current state and the probabilities of the future states.

Markov...