Exercise

1. Come up with a situation around you that can be defined as an MDP problem.
2. Do you think we can use the state-value function to solve the MDP problem in the same way as we use the action-value function?
3. Explore the active-function result by changing the following hyperparameters for the four-states MDP introduced here:
   1. Discount ratio
   2. Learning rate
   3. Reward in the transition from state 2 to 3

4. Use the following policy for the four-states MDP introduced in the chapter:
   1. Always choose action 1.
   2. Always choose action 2.
   3. Choosing the action maximizing the action value.
5. Try to run the CartPole example in the example code and see how the behavior is changed.