The A2C algorithm

The difference between A2C and A3C is that A2C performs synchronous updates. Here, all the workers will wait until they have completed the collection of experiences and computed the gradients. Only after this are the global (or master) network's parameters updated. This is different from A3C, where the update is performed asynchronously, that is, where the worker threads do not wait for the others to finish. A2C is easier to code than A3C, but that is not undertaken here. If you are interested in this, you are encouraged to take the preceding A3C code and convert it to A2C, after which the performance of both algorithms can be compared.