Markov chains
In our example, we did not use any strategy to maximize wins by the computer. However, let's imagine that a human does not just randomly choose his move, but his choice depends on his last move. Thus, we obtain a classical Markov chain with a finite state and discrete time. At every step, we can compute a new transition matrix based on the history of previous moves. Then, we choose the move that has the maximum probability, and we suppose that this will be a human's move. Based on this, the computer chooses a move to win against a human.
Let's consider step by step all the components of this approach. For a start, let's look at how the Markov chains are represented in Mathematica.
The DiscreteMarkovProcess function describes a time series whose elements constitute a discrete Markov chain. The first parameter of this function is the initial state of the chain, and the second parameter is the matrix of transition probabilities:
Using the MarkovProcessProperties function, we can...
