What is multi-agent RL?

The multi-agent setup is a natural extension of the familiar RL model that we covered in Chapter 1. In the classical RL setup, we have one agent communicating with the environment using observations, rewards, and actions. But in some problems that often arise in real life, we have several agents involved in the environment interaction. To give some concrete examples:

• A chess game, when our program tries to beat the opponent

• A market simulation, like product advertisements or price changes, when our actions might lead to counter-actions from other participants

• Multiplayer games, like Dota 2 or StarCraft II, when the agent needs to control several units competing with other players’ units (in this scenario, several units controlled by a single player might also cooperate to reach the goal)

If other agents are outside of our control, we can treat...