Chapter 4, Teaching a Smartcab to Drive Using Q-Learning

1. The term state is commonly used in the terminology of solving Markov decision processes, and it refers to the same entity as an observation in RL spaces. When describing MDPs specifically, it is consistent with the existing terminology to use the word state.
2. We know the Q-function has converged when updates to the Q-table no longer change its values.
3. The Q-table remains at its current values when the Q-function has converged.
4. When the Q-function has converged, meaning updates to the Q-table no longer cause its values to change, we know that the agent has found the optimal path to the goal.

5. The function numpy.argmax() returns the index of the maximum element in an array.
6. The function numpy.max() returns the value of the maximum element in an array.
7. The randomly-acting agent cannot learn from its actions. The Q-learning...