What is DevOps?

If you were to list all of the different definitions and descriptions of DevOps, there would be many. However, as different as these might be, they will most likely share several concepts. These are collaboration, continuous delivery of business value, and breaking down silos.

With all of the technical discussion in the rest in this book, it is important not to overlook the value proposition for adopting DevOps, namely, that it will help you to improve the way that you continuously deliver value to your end users. To do this, you have to decrease the time between starting work on a new feature and the first user using it in production. This means that you not only have to write the software but also deliver and operate it.

Over the last decade, the way we write software has fundamentally changed. More and more companies are now adopting an Agile way of working to increase the efficiency of their software development. More and more teams are now working in short iterations or sprints to create new increments of a product in quick succession. However, creating potentially shippable increments faster and faster does not create any value in itself. Only when each new version of your software is also released to production and used by your end users does it start delivering value.

In traditional organizations, developers and operators are often located in different departments and taking software into production includes a hand-off, often with a formal ceremony around it. In such an organization, it can be hard to accelerate that delivery to production along with the speed at which development can create new versions.

Next to that, development and operations departments often have conflicting goals. While a development department is rewarded for creating many changes as fast as possible, operation departments are rewarded for limiting downtime and preventing issues. The latter is often best achieved by having as few changes as possible. The conflict here is clear—both departments have optimizations for one subgoal, as shown in the following diagram:

This defeats the purpose of these subgoals, which comes from the shared, overarching goal of quickly taking in new versions while maintaining stability. Precisely this conflict between developmental and operational goals is one of the things that should disappear in a DevOps culture. In such a culture, developers and operations teams should work together on delivering new versions to production in a fast and reliable manner and share responsibility for both subgoals.

While it is good to know that DevOps is a cultural movement, tools and automation are an important part of that culture. In this book, the focus will be on these tools and how to use them to implement many of the practices that come with a DevOps culture. In other words, this book will be mostly on the products and processes associated with DevOps. If you want to learn more about the cultural side of things, about the people, there are many other books to read.

The rest of this section will explore the relation between DevOps to see how they complement each other. The focus will be on Agile techniques and prices for work management. We will also discuss the goals and benefits of a DevOps culture.

The relation between DevOps and Agile

If you take a look at Agile, you might notice that part of it is the focus on business value and shortening the time between the delivery of a new business value. From that perspective, adopting DevOps is a logical next step after Agile. Agile advocates that the software development teams' responsibilities should extend forward by engaging with users and other stakeholders to more quickly deliver valuable potentially shippable products. DevOps is all about not just creating something that might be shipped, but really shipping it as well. With Agile and DevOps combined, you can create an end-to-end, continuous flow of value to your users.

One of the things you need to be able to do this is a common approach to managing the work to be done for everyone involved. In the next section, you will find some pointers on how to incorporate operational concerns in the way you manage your work.

Agile work management

When you are starting to increase the collaboration between development and operations, you will quickly notice that they have to cope with different types of work. In development, a large part of the work is planned: user stories and bugs that are picked up from a backlog. On the other hand, for operations, a large part of their work is unplanned. They respond to warnings and alerts from systems and requests or tickets from users or developers.

Integrating these two, especially if developers and operators are located on the same team, can be challenging. To see how you can deal with this, let's explore the following approach:

1. First, switch to a flow-based way of working for developers.
2. Next, allow for operations to also list their work in the same work management system as developers using synchronizations. You can also choose to implement fastlaning, an approach to expedite urgent work.
3. Finally, you might choose to decommission existing ticketing tools for operations if possible.

Fastlaning is an approach to organizing work that allows for both planned and unplanned work by visualizing two separate lanes of work. To do this, the Scrum board is extended with a Kanban-like board on the top. This is the fast lane. On the Kanban board, urgent but unplanned work is added. Any work added to this lane is picked up by the team with the highest priority. Only when there is no work remaining in the fast lane is work from the Scrum board with planned work picked up. Whenever new work is added to the fast lane, this takes priority again. Often, there is the agreement though that work in progress is finished before switching to work in the fast lane.

Switching to a flow-based methodology

The first thing to consider is transitioning the way developers work from batch-wise to flow-based. An example of a batch-wise way of working is Scrum. If you are using the Scrum framework, you are used to picking up a batch of work every two to four weeks and focus on completing all of that work within that time window. Only when that batch is done do you deliver a potentially shippable product.

When changing to a flow-based approach, you try to focus not on a batch, but on just one thing only. You work on that one work item and drive it completely until it's done before you start on the next. This way, there is no longer a sprint backlog, only a product backlog. The advantage of this approach is that you no longer decide which work to perform upfront, but whenever you are free to start on new work, you can pick up the next item from the backlog. In an environment where priorities quickly shift, this allows you to react to change more quickly.

These changes to the way developers organize their work make it easier to include operations in work management, but there is also another benefit. When developers are focusing on getting a single work item done instead of a whole sprint at once, you can also increase the number of times you can deliver a small portion of value to your users.

Synchronizing work items to one system

After the development team changes the way it organizes its work, it should now be easier for developers to also list their planned work on the shared backlog and pull work from that backlog when they have time to work on it. They now also have a place where they can list their unplanned work.

However, there might still be an existing ticketing system where requests for operations are dropped by users or automatically created by monitoring tools. While Azure DevOps has a great API to rework this integration to directly create work items in Azure DevOps, you might first choose to create a synchronization between your existing ticketing tool and Azure Boards. There are many integration options available and there is a lot of ongoing work in this area. This way, operators can slowly move from their old tool to the new one, since they are now in sync. Of course, the goal is for them to move over to the same tool as the developers completely.

Fastlaning

With the work of developers and operators in the same work management tool, you will notice that you have a mix of planned and unplanned, often urgent, work in the system. To ensure that urgent work gets the attention and precedence it deserves, you can introduce what is called a fastlane to your sprint board. In the following screenshot, you can see an example of an Azure Board that is set up for fastlaning production issues:

The use of this horizontal split in the board is to only work on tasks in the regular lane when there is no work to be picked up in the fast lane.

Decommissioning other work management tools

After creating a shared work management system between development and operations, there is much opportunity to increase the amount of collaboration between them. When this collaboration is taking off, old ticketing systems that were used by operations might now slowly be decommissioned over time. Integrations from monitoring tools can be shifted to the new shared tools and the number of tickets between developers and operators should slowly decrease as they find new ways of working together.

Goals and benefits of a DevOps culture

At this point, you might be wondering about the point of it all. What are the benefits of DevOps and what is there in it for you, your colleague, and your organization? The most common goal of adopting DevOps is to achieve a reduction in cycle time. Cycle time is the time between starting work on a new feature and the moment that the first user can use it. The way this is achieved, by automation, also serves the goals of lower change failure rate, lower Mean Time To Repair (MTTR) and lower planned downtime.

Next to all that, there might also be other benefits such as increased employee satisfaction, less burnout and stress, and better employee retention. This is attributed to the removal of opposing goals between developers and operators.

For a while, there was doubt whether DevOps really works, and whether these goals were really met, and whether the extra benefits were really achieved, as this was only shown using case studies. The downside of this is that case studies are often only available for successful cases and not for unsuccessful cases. This all changed in 2018 when the book Accelerate came out. This book shows, based on years of quantitative research, that modern development practices such as DevOps really contribute to reaching IT goals and organizational goals.

Measuring results

To measure where you currently stand as a team or organization and the impact of DevOps on you, there are several metrics that you could start recording. As always when working with metrics or Key Performance Indicators (KPIs), make sure that you do not encourage people to game the system by looking only at the numbers. Several interesting metrics are detailed in the following sections and if you go over them, you will notice that they are all about encouraging flow.

Cycle time and lead time

Cycle time and lead time are metrics that come from Lean and Kanban and are used to measure the time needed to realize a change. Cycle time is the amount of time between starting work on a feature and users being able to use that feature in production. The lower cycle time, the quicker you can react to changing requirements or insights. Lead time is the amount of time between requesting a feature and realizing that feature. It is the time between adding work to the backlog and the start of implementation.

When you add cycle time and lead time together, you are calculating another metric, the time to market. This last one is often an important business metric when developing software. Minimizing both cycle time and lead time will hence have a business impact.

The amount of work in progress

Another thing you can measure is the amount of work in progress at any point in time. DevOps is focusing on the flow of value to the user. This implies that everyone should, if possible, be doing only one thing at a time and completely finish that before moving on to something else. This reduces the amount of time spent on task switching and the amount of time spent on not yet complete work. Measuring how many things a team works on in parallel and reporting on this can encourage this.

You can even go as far as putting actual limits on the amount of work that can be in progress. The following is a small part of the earlier screenshot, showing that these work-in-progress limits can even be shown in the tool:

The goal is to have as little work in progress at the same time as possible.

Mean time to recovery

A third metric is the mean time to recovery. How long does it take you to restore service in case of a (partial) outage? In the past, companies focused on reducing the mean time between failures. This used to be the mean indicator of the stability of a product. However, this metric encourages limiting the number of changes going to production. The unwanted consequence often is that outages, though maybe rare, last long and are hard to fix.

Measuring the mean time to recovery shifts the attention to how quickly you can remediate an outage. If you can fix outages quickly, you achieve the same, namely, minimizing the amount of downtime without sacrificing the rate of change. The goal is to minimize the time to recovery.

Change rate and change failure rate

Finally, you can measure the number of changes delivered to production and the percentage of that which is not successful. Increasing the rate of change implies that you are more often delivering value to your users, hence realizing a flow of value. Also, by measuring not just the number of failures, but the percentage that fails, you are actually encouraging many small, successful changes instead of encouraging limiting the number of changes overall.

Your goals should be to increase the rate of change while lowering the change failure rate.

At this point, you might be wondering, how do I change my organization to foster this culture and reap all of these benefits? The next section will answer this for you.