Startup and shutdown
One vital test for your application is how well it starts up and shuts down. While your system might work well running in a steady state, how long does it take to get there, and can it start quickly when under load? For an operations team, the ability to restart a system is an emergency fallback to recover from unknown states, so it has to work. Worse, applications might crash at any time, and you need to know they can recover without manual intervention, which would make outages stretch into minutes or hours instead of seconds.
For each release, check how long your system takes to start up. Have any inefficiencies or extra processes been introduced? As with many gray areas, you’ll have to work with the product owner to decide the time limit. There are two cases to consider – one where your whole application or one subsystem is restarting without any load. The other is to restart one part of your system while it is loaded. Measure the performance...