The traditional change management process

For engineers who have worked in a large network environment, they know that the impact of a network change gone wrong can be big. We can make hundreds of changes without any issues, but all it takes is one bad change that can cause the network to have a negative impact on the whole business.

There is no shortage of war stories about network outages causing business pain. One of the most visible and large-scale AWS EC2 outages in 2011 was caused by a network change that was part of the normal AWS scaling activities in the AWS US-East region. The change occurred at 00:47 PDT and caused a brown-out for various services for over 12 hours, losing millions of dollars for Amazon in the process. More importantly, the reputation of the relatively young service took a serious hit. IT decision-makers pointed to the outage as reasons to NOT migrate to the young AWS cloud. It took many years to rebuild its reputation. You can read more...