Before deep diving into the first RL algorithm that solves the optimal Bellman equation, we want to give a broad but detailed overview of RL algorithms. We need to do this because their distinctions can be quite confusing. There are many parts involved in the design of algorithms, and many characteristics have to be considered before deciding which algorithm best fits the actual needs of the user. The scope of this overview presents the big picture of RL so that in the next chapters, where we'll give a comprehensive theoretical and practical view of these algorithms, you will already see the general objective and have a clear idea of their location in the map of RL algorithms.
The first distinction is between model-based and model-free algorithms. As the name suggests, the first requires a model of the environment, while the second is free from...
