Why a continuous space?

All the examples that we've seen so far in the book had a discrete action space, so you might have the wrong impression that discrete actions dominate the field. This is a very biased view, of course, and just reflects the selection of domains that we picked our test problems from. Besides Atari games and simple, classical RL problems, there are lots of tasks requiring more than just making a selection from a small and discrete set of things to do.

To give you an example, just imagine a simple robot with only one controllable joint, which can be rotated in some range of degrees. Usually, to control a physical joint, you have to specify either the desired position or the force applied. In both cases, you need to make a decision about a continuous value. This value is fundamentally different from a discrete action space, as the set of values that you can make a decision on is potentially infinite. For instance, you can ask the joint to move to a...