Questions

1. Why do we choose to use the words state and observation interchangeably? When would be a more appropriate time to use the word state?
2. How do we know when the Q-function has converged?
3. What happens to the Q-table when the Q-function has converged?
4. When do we know the agent has found the optimal path to the goal? Describe in terms of the previous two questions.
5. What does numpy.argmax() return?
6. What does numpy.max() return?
7. Why does the randomly-acting agent take thousands of time steps to reach the goal? How does the Q-learning agent perform better?
8. Describe one benefit of decaying alpha.
9. What is overfitting and how does it apply in the context of an RL model?
10. By what order of magnitude does the number of time steps needed to reach the goal reduce when the number of training episodes is multiplied by 10? Give a general response to this; there may be multiple valid...