Comparing model-based and model-free methods
In Chapter 4, we saw several different ways in which we can classify RL methods. We distinguished three main categories:
-
Value-based and policy-based
-
On-policy and off-policy
-
Model-free and model-based
So far, we have covered enough examples of methods of both types in the first and second categories, but all the methods that we have covered so far have been 100% model-free. However, this doesn’t mean that model-free methods are more important or better than their model-based counterparts. Historically, due to their sample efficiency, model-based methods have been used in the robotics field and for other industrial controls. This has also happened because of the cost of the hardware and the physical limitations of samples that can be obtained from a real robot. Robots with a large degree of freedom are not widely accessible...
