Chapter 5

1. In Q-learning, what does the Q stand for (you will have to research this on the internet)?

   The origin of Q-learning is the doctoral thesis of Christopher John Cornish Hellaby Watkins from King’s College, London, May, 1989 (https://www.researchgate.net/publication/33784417_Learning_From_Delayed_Rewards). Evidently, the Q just stands for Quantity.

2. What could we do to limit the number of states that the Q-learning algorithm must search through?

   Only pick the Q-states that are relevant and are follow-ons to the current state. If one of the states is impossible to reach from the current position, or state, then don’t consider it.

3. What effect does changing the learning rate have on the learning process?

   If the learning rate is too small, the training can take a very long time. If the learning rate is too large, the system does not learn a path but instead overshoots and may miss the minimum or optimum solution. If the learning rate is too big, the solution...