Multi-Agent RL

In the last chapter, we discussed discrete optimization problems. In this final chapter, we will introduce multi-agent reinforcement learning (sometimes abbreviated to MARL), a relatively new direction of reinforcement learning (RL) and deep RL, which is related to situations when multiple agents communicate in an environment. In real life, such problems appear in auctions, broadband communication networks, Internet of Things, and other scenarios.

In this chapter, we will just take a quick glance at MARL and experiment a bit with simple environments; but, of course, if you find it interesting, there are lots of things you can experiment with. In our experiments, we will use a straightforward approach, with agents sharing the policy that we are optimizing, but the observation will be given from the agent’s standpoint and include information about the other agent’s location. With that simplification, our RL methods will stay the same, and...