Managing a tiny forest problem
As we mentioned in Chapter 5, Simulation-Based Markov Decision Processes, a stochastic process is called Markovian if it starts from an instant t in which an observation of the system is made. The evolution of this process will depend only on t, so it will not be influenced by the previous instants. So, a process is called Markovian when the future evolution of the process depends only on the instant of observing the system and does not depend in any way on the past. MDP is characterized by five elements: decision epochs, states, actions, transition probability, and reward.
Summarizing the Markov decision process
The crucial elements of a Markovian process are the states in which the system finds itself, and the available actions that the decision maker can carry out on that state. These elements identify two sets: the set of states in which the system can be found, and the set of actions available for each specific state. The action chosen by...
