Using an outage to test migration
While planned migrations are always preferred, sometimes hardware failures or server instability will introduce an aspect of surprise. If we had not used Pacemaker, a server crash would be a catastrophic event. Even if we had followed every chapter in this book this far and had Nagios and e-mail alerts galore, a DBA would need to be available to activate the alternate node.
If an outage occurred at night when everyone was sleeping, we would be faced with a worst-case scenario. Necessary personnel might not hear the alert for several minutes, and more time is lost on triage and activation steps. Such an outage could extend from a few minutes to over an hour. So much for our high availability!
Yet, at this point, we don't know if Pacemaker would negate the previous scenario. While we've tested how Pacemaker handles an expected and safe migration, what happens when a node disappears entirely? Will Pacemaker cover us in the event there is an outage when nobody...
