Reconciliation

Mesos implements an actor-style message passing programming model to enable nonblocking communication between different Mesos components and leverages protocol buffers for the same. For example, a scheduler needs to tell the executor to utilize a certain number of resources, an executor needs to provide status updates to the scheduler regarding the tasks that are being executed, and so on. Protocol buffers provide the required flexible message delivery mechanism to enable this communication by allowing developers to define custom formats and protocols, which can be used across different languages.

An at-most-once message delivery model is employed for this purpose except for certain messages, such as status updates, a lot of which follow the at-least-once delivery model by making use of acknowledgements. In case of failures, there is a high chance that messages between the master and slaves can get lost leading to state inconsistencies.

For instance, there are multiple scenarios...