What shall we do with the stateful services inside our cluster? If any of them fails and Swarm reschedules it, the state will be lost. Even if impossible happens and none of the replicas of the service ever fail, sooner or later we'll have to upgrade the cluster. That means that existing nodes will be replaced with new images and Swarm will have to reschedule your services to the new nodes. In other words, services will fail or be rescheduled, and we might need to persist state when they are stateful.
Let us go through each of the stateful services we're currently running inside our cluster.
The obvious case of stateful services is databases. We are running MongoDB. Should we persist its state? Many would answer positively to that question. I'll argue against persisting data on disk. Instead, we should create a replica set with at least three MongoDBs...
