Understanding the process aspect of DevOps

Whenever DevOps is discussed, it is tempting to make technology or tooling the main focus. However, without well-defined processes in place, any benefits or results achieved from adopting DevOps will be limited at best, and it may even create additional challenges and complexities!

In the DevOps methodology, the process aspect refers to the creation of an efficient and streamlined workflow for software development, testing, and deployment. The goal is to optimize the development process to ensure that software is delivered quickly and reliably to end users while maintaining a high level of quality.

This involves the use of agile development methodologies and continuous integration and continuous delivery (CI/CD) practices. These practices involve automating various aspects of the software development lifecycle, such as code testing, building, and deployment. Generally, when an organization adopts a DevOps approach, it must implement five essential practices: Agile Planning, Version Control, Continuous Integration (CI), Continuous Delivery (CD), and Continuous Monitoring (see Figure 1.1):

Figure 1.1 – The five essential practices of DevOps

It is worth noting that these are not the only practices in DevOps, but they are considered to be crucial ones. In the next section, we will describe these five core practices in more detail.

Important note

For those keen on exploring other definitions and models related to DevOps, the DevOps Competence Model by the DevOps Agile Skills Association (DASA) is a valuable resource. You can find more information about it here: https://www.dasa.org/products/guidance-products/team-competence-model/.

Understanding the five core practices of DevOps

In this section, we will examine the five fundamental practices of DevOps, beginning with agile planning.

Agile planning is a broad reference to techniques used to plan and track our software projects in DevOps. It is a project management approach that involves breaking down a project into small, manageable pieces and working on them iteratively. The agile methodology was formally launched in 2001 through the Agile Manifesto, covering the main principles of Agile project management. To get more information on the Agile Manifesto, you can go to https://agilemanifesto.org/.

The goal is to deliver a functional product incrementally and continuously while taking feedback from the stakeholders.

A simple example of agile planning can be seen in the development of a mobile app. Let’s say a company wants to develop a mobile application that can be used to order food from local restaurants. The development team would first identify the key features that the app should have, such as a menu, ordering system, payment system, and user profiles. With these requirements in hand, they would then design the architecture of the app. Following this, the team would break down these features into smaller, more manageable tasks, such as designing the user interface, creating a database to store orders, and integrating the payment system. The team would then prioritize these tasks based on the business value they add and the level of effort required to complete them. Once the tasks are prioritized, the team would estimate the time required to complete each task and create a sprint plan. A sprint is a short, time-boxed period (usually 1–2 weeks) during which the team works on a set of tasks.

During each sprint, the team would work on the tasks in priority order, complete them, and get feedback from stakeholders. The feedback would then be used to make adjustments to the product and the plan for the next sprint. This process of breaking down tasks, prioritizing them, estimating time, and working iteratively with feedback is the core of agile planning.

Important note

To understand the guiding values of agile development, we recommend reviewing the twelve principles of agile development that are highlighted here: https://www.agilealliance.org/agile101/12-principles-behind-the-agile-manifesto/.

The second practice, version control, allows developers to manage changes to code efficiently, collaborate effectively, and keep track of all changes made to the code. Figure 1.2 shows a simple example of how version control works in DevOps. Suppose a team of developers is working on a software application. They create a repository (a central location to store code) using a version control system (VCS) such as Git. Each developer can clone the repository to their local computer, or they might work directly in a controlled development environment, eliminating the need to copy code to a local PC. It is worth noting that some companies have strict policies regarding this workflow and do not allow code to be cloned locally.

Figure 1.2 – Version control and branching example

Let’s say Developer A is assigned to work on feature A; they create a new branch in the repository called feature A and start making changes to the code. Meanwhile, Developer B is working on a different feature in the application. They create a new branch called feature B and start making changes to the code. Both developers can work on their features independently without affecting each other’s work. Once they have completed their changes, they can merge their branches back into the main branch (also called the trunk branch) in the repository.

If there are any conflicts between the changes made by the two developers, the VCS will highlight them, and the developers can resolve them before merging the branches. The VCS also keeps a record of all changes made to the code, including who made them, when they were made, and why they were made. If there is a problem with the new code, the team can use the VCS to roll back to a previous version of the code quickly. This rollback feature is useful if a bug is introduced into the code or if the new changes cause unexpected problems.

The third practice, continuous integration (CI), refers to the ongoing validation of code quality whenever developers contribute or modify code. Suppose a team of developers is working on a software project; each time a developer finishes making changes to their code and commits those changes to the shared repository, an automatic process is triggered on a CI server, such as Jenkins or Travis CI, to build the software, run unit tests, and check for code quality issues using various tools. If the build and tests pass successfully, the CI server will notify the team that the changes are ready for review and integration. If any errors or issues are detected, the CI server will alert the team, and they can then work together to fix the issues before merging the code into the shared repository. This allows the team to catch and fix issues early in the development cycle, reducing the risk of bugs and errors in the final product:

Figure 1.3 – Sample CI flow

The fourth practice, continuous delivery (CD), refers to the ongoing testing and deployment of validated software using an automated process. It allows teams to release new features and bug fixes quickly using a continuous process. The goal of CD is to enable development teams to deliver software changes to production quickly and with confidence while maintaining a high level of quality and reliability.

Suppose a team of developers is working on a web application; when the team writes code for a new feature, it is committed to a version control system and is automatically tested by a series of automated tests, including unit tests, integration tests, and acceptance tests. Once the code passes all the tests, it’s automatically deployed to a staging environment where it undergoes additional testing and review by the product owner. If everything looks good, the code is then automatically deployed to production, where it’s made available to all users.

The fifth practice of continuous monitoring involves gathering feedback from users and collecting telemetry data from running applications in real time. The goal is to ensure that software systems are meeting the needs of users and delivering value to the organization. It requires gathering continuous insights into the performance and behavior of software systems and using that information to make data-driven decisions that improve the overall quality and user experience. To understand this practice better, let’s break it down into two components:

• Gathering feedback from users: User feedback is an essential component of continuous monitoring because it helps to identify issues and areas for improvement in the software system from the user’s perspective. Feedback can be collected through various channels, such as surveys, user reviews, and support tickets. By analyzing this feedback, development teams can identify patterns and trends that highlight areas for improvement and prioritize these improvements based on their impact on the user experience.
• Collecting telemetry data from running applications: Telemetry data refers to a broad range of information collected from various sources as the software system operates in real time. These sources can include application logs, server metrics, network traffic, user interactions, error reports, and more. Metrics such as response times, error rates, and server load, as well as insights into user behavior, can be derived from these data. By collecting and analyzing telemetry data, development teams can gain a comprehensive understanding of the software’s performance and user interactions. This data is invaluable for detecting anomalies and potential issues before they escalate into critical problems.

By combining user feedback with telemetry data, development teams can gain a comprehensive understanding of how the software system is performing and how it is being used. This information can then be used to make data-driven decisions about how to improve the system and prioritize future development efforts. Overall, the fifth practice of continuous monitoring is a crucial part of DevOps that helps to ensure that software systems meet the needs of users and deliver value to the organization.

Understanding the stages in a DevOps workflow

Understanding the five essential practices of DevOps is vital, but how do organizations put them into action? The implementation of DevOps practices involves a set of stages that facilitate the constant development, testing, and deployment of software. These stages may differ based on the organization and the type of software being developed, but they typically follow the pattern shown in Figure 1.4:

Figure 1.4 – Typical stages in a DevOps workflow

The first stage is Plan, where the agile planning practice is put into action. At this stage, teams plan and prioritize what needs to be accomplished based on business or customer requirements. This involves creating a project plan or roadmap, researching to understand the required architectural changes, defining the scope of work (such as feature development or bug fixing), breaking down the plan into smaller and assignable tasks, estimating the time required for each task, and setting priorities for the tasks that need to be completed first.

The second stage is Code, which involves the actual coding and development of software using the selected programming languages, frameworks, and tools. It is at this stage that version control practices are implemented. The team collaborates to develop the code and commit changes to a version control system.

The third stage is Build and Test, where continuous integration practices are implemented. In this stage, the code is converted into executable software and tested to guarantee that it works as intended and fulfills project requirements. A combination of automated and manual tests is employed to detect and resolve any errors, bugs, or defects.

The fourth stage is Release and Deploy, where the software is packaged and released into the production environment. This is where continuous delivery practices are implemented. This stage involves setting up the infrastructure required to run the software and configuring it to work, deploying the software into a pre-production environment to run additional validation, and deploying validated software into production.

The fifth stage is Operate and Monitor, where the software is actively monitored and maintained. The team watches for any issues or incidents after deployment, examining the application’s performance, collecting and analyzing logs, and ensuring that the software complies with defined service level agreements (SLAs). In this stage, continuous monitoring tools and practices are used to track the application’s performance, gather usage telemetry and performance metrics, and detect any potential issues before impacting users. The gathered information is then used to identify areas for optimization or additional features to be added. A self-healing approach that leverages automation is increasingly popular at this stage. This approach involves using automation to correct any failures or errors without requiring human intervention, such as terminating a problematic application instance and deploying a replacement instance or triggering failover to a passive instance in the case of unexpected events. Implementing this approach significantly improves system availability and reliability and enables faster and more efficient recovery from failures.

These stages form a continuous cycle that empowers teams to continuously deliver value to end users while enhancing their software development procedures. Keep in mind that speed is crucial to a successful DevOps workflow! It is essential that each stage is executed quickly and efficiently (we will revisit this aspect when we talk about security integrations).