To follow along with the instructions in this chapter, you will need the following:

• A PC with an internet connection
• A valid email address

If you ask 10 people what DevOps is, you will probably get 10 different answers, depending on these people’s backgrounds and probably the books they have read. Therefore, it is important for us to establish a working definition that we will use for DevOps for the rest of this book. Microsoft’s official definition of DevOps was coined by Donavan Brown at a conference in 2018. You can still find the video on YouTube: https://www.youtube.com/watch?v=cbFzojQOjyA. Here is the definition:

DevOps is the union of people, process, and products to enable continuous delivery of value to our end users.

From this definition, we want to highlight a few essential points. To start with, it is essential to understand that DevOps is not a tool, a product, or a job title. Instead, it is a collaborative approach to software development. It is a way of working/thinking, and most of all, it is a change of culture (more on this later). Another key point to note is that the primary goal of DevOps is to ensure the speedy and frequent delivery of functional software to end users. If what has been implemented does not have this impact, it is likely not DevOps, or it has not been appropriately implemented (we will discuss this in more detail in the Staying clear of DevOps anti-types section in this chapter). The last point that we would like to stress is that there are three aspects to DevOps. There is a people aspect, a process aspect, and a product aspect. In the next section, we will begin by examining the process aspect, but before we do that, let’s discuss why organizations are rapidly moving towards a DevOps approach for software development and delivery.

The why of DevOps – Innovation, velocity, and speed

While we have dedicated significant time to discussing the process, people, and product aspects of DevOps, it is equally important to understand the driving factors that lead companies to embrace DevOps and the reasons for its growing significance in recent years. DevOps provides unique advantages to companies that other software delivery approaches cannot match. The following points are some of the benefits associated with DevOps adoption:

• Accelerating time to market: This refers to the ability to bring new products to market faster. According to research conducted by Puppet, companies that embrace the culture and practices of DevOps deploy code 46 times more frequently compared to those that do not.
• Adapting to the market and competition: This means being able to adapt to changes in the market and competition. For example, Etsy, an online marketplace for handmade and vintage goods, uses DevOps practices to deploy code changes 50 times per day. This allows the company to quickly test and launch new features, respond to user feedback, and stay ahead of competitors.
• Maintaining system stability and reliability: DevOps practices can help organizations maintain system stability and reliability by improving communication and collaboration between development and operations teams. For example, Netflix uses a DevOps approach to ensure that its streaming service remains available and responsive at all times. The company achieves this by automating its infrastructure deployment and using a “chaos monkey” tool to intentionally introduce failures in its systems, which helps identify and address weaknesses before they cause problems.
• Improving mean time to recovery: By adopting DevOps practices, organizations can improve their ability to recover from incidents and outages more efficiently. For instance, Target, a leading retail company in the US, reduced its overall mean time to recovery (MTTR) by 90% after implementing DevOps practices. This allowed the company to minimize the impact of outages and maintain high levels of customer satisfaction.

With the basics covered, let’s delve into the process used in DevOps to create workflows.

Whenever DevOps is discussed, it is tempting to make technology or tooling the main focus. However, without well-defined processes in place, any benefits or results achieved from adopting DevOps will be limited at best, and it may even create additional challenges and complexities!

In the DevOps methodology, the process aspect refers to the creation of an efficient and streamlined workflow for software development, testing, and deployment. The goal is to optimize the development process to ensure that software is delivered quickly and reliably to end users while maintaining a high level of quality.

This involves the use of agile development methodologies and continuous integration and continuous delivery (CI/CD) practices. These practices involve automating various aspects of the software development lifecycle, such as code testing, building, and deployment. Generally, when an organization adopts a DevOps approach, it must implement five essential practices: Agile Planning, Version Control, Continuous Integration (CI), Continuous Delivery (CD), and Continuous Monitoring (see Figure 1.1):

Figure 1.1 – The five essential practices of DevOps

It is worth noting that these are not the only practices in DevOps, but they are considered to be crucial ones. In the next section, we will describe these five core practices in more detail.

Important note

For those keen on exploring other definitions and models related to DevOps, the DevOps Competence Model by the DevOps Agile Skills Association (DASA) is a valuable resource. You can find more information about it here: https://www.dasa.org/products/guidance-products/team-competence-model/.

Understanding the five core practices of DevOps

In this section, we will examine the five fundamental practices of DevOps, beginning with agile planning.

Agile planning is a broad reference to techniques used to plan and track our software projects in DevOps. It is a project management approach that involves breaking down a project into small, manageable pieces and working on them iteratively. The agile methodology was formally launched in 2001 through the Agile Manifesto, covering the main principles of Agile project management. To get more information on the Agile Manifesto, you can go to https://agilemanifesto.org/.

The goal is to deliver a functional product incrementally and continuously while taking feedback from the stakeholders.

A simple example of agile planning can be seen in the development of a mobile app. Let’s say a company wants to develop a mobile application that can be used to order food from local restaurants. The development team would first identify the key features that the app should have, such as a menu, ordering system, payment system, and user profiles. With these requirements in hand, they would then design the architecture of the app. Following this, the team would break down these features into smaller, more manageable tasks, such as designing the user interface, creating a database to store orders, and integrating the payment system. The team would then prioritize these tasks based on the business value they add and the level of effort required to complete them. Once the tasks are prioritized, the team would estimate the time required to complete each task and create a sprint plan. A sprint is a short, time-boxed period (usually 1–2 weeks) during which the team works on a set of tasks.

During each sprint, the team would work on the tasks in priority order, complete them, and get feedback from stakeholders. The feedback would then be used to make adjustments to the product and the plan for the next sprint. This process of breaking down tasks, prioritizing them, estimating time, and working iteratively with feedback is the core of agile planning.

Important note

To understand the guiding values of agile development, we recommend reviewing the twelve principles of agile development that are highlighted here: https://www.agilealliance.org/agile101/12-principles-behind-the-agile-manifesto/.

The second practice, version control, allows developers to manage changes to code efficiently, collaborate effectively, and keep track of all changes made to the code. Figure 1.2 shows a simple example of how version control works in DevOps. Suppose a team of developers is working on a software application. They create a repository (a central location to store code) using a version control system (VCS) such as Git. Each developer can clone the repository to their local computer, or they might work directly in a controlled development environment, eliminating the need to copy code to a local PC. It is worth noting that some companies have strict policies regarding this workflow and do not allow code to be cloned locally.

Figure 1.2 – Version control and branching example

Let’s say Developer A is assigned to work on feature A; they create a new branch in the repository called feature A and start making changes to the code. Meanwhile, Developer B is working on a different feature in the application. They create a new branch called feature B and start making changes to the code. Both developers can work on their features independently without affecting each other’s work. Once they have completed their changes, they can merge their branches back into the main branch (also called the trunk branch) in the repository.

If there are any conflicts between the changes made by the two developers, the VCS will highlight them, and the developers can resolve them before merging the branches. The VCS also keeps a record of all changes made to the code, including who made them, when they were made, and why they were made. If there is a problem with the new code, the team can use the VCS to roll back to a previous version of the code quickly. This rollback feature is useful if a bug is introduced into the code or if the new changes cause unexpected problems.

The third practice, continuous integration (CI), refers to the ongoing validation of code quality whenever developers contribute or modify code. Suppose a team of developers is working on a software project; each time a developer finishes making changes to their code and commits those changes to the shared repository, an automatic process is triggered on a CI server, such as Jenkins or Travis CI, to build the software, run unit tests, and check for code quality issues using various tools. If the build and tests pass successfully, the CI server will notify the team that the changes are ready for review and integration. If any errors or issues are detected, the CI server will alert the team, and they can then work together to fix the issues before merging the code into the shared repository. This allows the team to catch and fix issues early in the development cycle, reducing the risk of bugs and errors in the final product:

Figure 1.3 – Sample CI flow

The fourth practice, continuous delivery (CD), refers to the ongoing testing and deployment of validated software using an automated process. It allows teams to release new features and bug fixes quickly using a continuous process. The goal of CD is to enable development teams to deliver software changes to production quickly and with confidence while maintaining a high level of quality and reliability.

Suppose a team of developers is working on a web application; when the team writes code for a new feature, it is committed to a version control system and is automatically tested by a series of automated tests, including unit tests, integration tests, and acceptance tests. Once the code passes all the tests, it’s automatically deployed to a staging environment where it undergoes additional testing and review by the product owner. If everything looks good, the code is then automatically deployed to production, where it’s made available to all users.

The fifth practice of continuous monitoring involves gathering feedback from users and collecting telemetry data from running applications in real time. The goal is to ensure that software systems are meeting the needs of users and delivering value to the organization. It requires gathering continuous insights into the performance and behavior of software systems and using that information to make data-driven decisions that improve the overall quality and user experience. To understand this practice better, let’s break it down into two components:

• Gathering feedback from users: User feedback is an essential component of continuous monitoring because it helps to identify issues and areas for improvement in the software system from the user’s perspective. Feedback can be collected through various channels, such as surveys, user reviews, and support tickets. By analyzing this feedback, development teams can identify patterns and trends that highlight areas for improvement and prioritize these improvements based on their impact on the user experience.
• Collecting telemetry data from running applications: Telemetry data refers to a broad range of information collected from various sources as the software system operates in real time. These sources can include application logs, server metrics, network traffic, user interactions, error reports, and more. Metrics such as response times, error rates, and server load, as well as insights into user behavior, can be derived from these data. By collecting and analyzing telemetry data, development teams can gain a comprehensive understanding of the software’s performance and user interactions. This data is invaluable for detecting anomalies and potential issues before they escalate into critical problems.

By combining user feedback with telemetry data, development teams can gain a comprehensive understanding of how the software system is performing and how it is being used. This information can then be used to make data-driven decisions about how to improve the system and prioritize future development efforts. Overall, the fifth practice of continuous monitoring is a crucial part of DevOps that helps to ensure that software systems meet the needs of users and deliver value to the organization.

Understanding the stages in a DevOps workflow

Understanding the five essential practices of DevOps is vital, but how do organizations put them into action? The implementation of DevOps practices involves a set of stages that facilitate the constant development, testing, and deployment of software. These stages may differ based on the organization and the type of software being developed, but they typically follow the pattern shown in Figure 1.4:

Figure 1.4 – Typical stages in a DevOps workflow

The first stage is Plan, where the agile planning practice is put into action. At this stage, teams plan and prioritize what needs to be accomplished based on business or customer requirements. This involves creating a project plan or roadmap, researching to understand the required architectural changes, defining the scope of work (such as feature development or bug fixing), breaking down the plan into smaller and assignable tasks, estimating the time required for each task, and setting priorities for the tasks that need to be completed first.

The second stage is Code, which involves the actual coding and development of software using the selected programming languages, frameworks, and tools. It is at this stage that version control practices are implemented. The team collaborates to develop the code and commit changes to a version control system.

The third stage is Build and Test, where continuous integration practices are implemented. In this stage, the code is converted into executable software and tested to guarantee that it works as intended and fulfills project requirements. A combination of automated and manual tests is employed to detect and resolve any errors, bugs, or defects.

The fourth stage is Release and Deploy, where the software is packaged and released into the production environment. This is where continuous delivery practices are implemented. This stage involves setting up the infrastructure required to run the software and configuring it to work, deploying the software into a pre-production environment to run additional validation, and deploying validated software into production.

The fifth stage is Operate and Monitor, where the software is actively monitored and maintained. The team watches for any issues or incidents after deployment, examining the application’s performance, collecting and analyzing logs, and ensuring that the software complies with defined service level agreements (SLAs). In this stage, continuous monitoring tools and practices are used to track the application’s performance, gather usage telemetry and performance metrics, and detect any potential issues before impacting users. The gathered information is then used to identify areas for optimization or additional features to be added. A self-healing approach that leverages automation is increasingly popular at this stage. This approach involves using automation to correct any failures or errors without requiring human intervention, such as terminating a problematic application instance and deploying a replacement instance or triggering failover to a passive instance in the case of unexpected events. Implementing this approach significantly improves system availability and reliability and enables faster and more efficient recovery from failures.

These stages form a continuous cycle that empowers teams to continuously deliver value to end users while enhancing their software development procedures. Keep in mind that speed is crucial to a successful DevOps workflow! It is essential that each stage is executed quickly and efficiently (we will revisit this aspect when we talk about security integrations).

Simply implementing DevOps practices in a continuous workflow is insufficient to fully unlock its potential; a cultural component is also necessary. Implementing DevOps methodologies delivers better results in a culture that promotes communication, collaboration, and shared responsibility among the members of development and operations teams. However, for many organizations (particularly larger ones), this proves to be the most difficult aspect of embracing DevOps since it involves a change in mindset and company culture, which can challenge established policies and procedures that have yielded positive results thus far.

The importance of a collaborative culture

To realize the full potential of DevOps, an organization must embrace a collaborative culture! By this, we mean a culture that breaks down team silos and allows developers, operations engineers, and other stakeholders to work together to achieve the shared goal of continuously delivering high-quality software to customers. This can be achieved by creating cross-functional teams or vertical teams.

Traditionally, large organizations have organized their teams in a horizontal structure based on particular skill sets or functional areas such as development, testing, or operations (as shown in Figure 1.5). Each team concentrates on their area of expertise and only handles tasks within that domain. The teams are separated by a boundary (as illustrated in Figure 1.6.) and are measured using different performance metrics, which frequently results in conflicts.

Figure 1.5 – Team boundaries in software development

On the other hand, DevOps advocates for and flourishes in teams that are organized vertically around particular products or services, also known as cross-functional teams. This structure brings together individuals from diverse functional areas to collaborate on a common objective of delivering a specific product or service. Each team member possesses a wide range of skills and is responsible for contributing to the delivery of that product or service. The teams are also measured using a shared set of performance metrics, which encourages team members to leverage each other’s skills and expertise to achieve shared goals. For example, a vertical team may be composed of developers, testers, and operations engineers collaborating to deliver a specific application or service, as shown in the following figure:

Figure 1.6 – Vertical team boundaries

It is crucial to note that while the composition of teams is vital, the presence of a guiding figure, often a servant-leader type, is equally important. Teams require clear direction and leadership to function optimally. This leader ensures that the team remains aligned with its goals, facilitates collaboration, and provides the necessary support to address challenges.

There are other cultural components of DevOps, such as fostering a culture of continuous learning and experimentation, ownership, and accountability. However, we recommend reading The Phoenix Project by Gene Kim for a more detailed understanding of these components.

Staying clear of DevOps anti-types

When implementing a DevOps culture, it is important to be aware of potential anti-patterns and anti-types. These are ineffective and sometimes counterproductive approaches that can hinder the successful implementation of DevOps.

For example, in an effort to implement DevOps, a manager or executive may create a separate DevOps team, which can further divide development and operations teams (Figure 1.7). The only time this separation may make sense is when the team is temporary, with a clear mandate to bring the teams closer together:

Figure 1.7 – Anti-type pattern 1

Another common anti-type is when developers or development managers assume they can do without operational skills and activities (Figure 1.8). This misconception is often rooted in a misguided understanding of cloud computing, which assumes that the self-service nature of cloud computing makes operational skills obsolete. However, this perspective grossly underestimates the complexities and significance of operational skills and results in avoidable operational mistakes:

Figure 1.8 – Anti-type pattern 2

Yet another anti-type is when organizations simply rename their operations team as a DevOps or site reliability engineering (SRE) team without making any real change to their processes or silos (refer to Figure 1.9). This approach fails to understand or appreciate the importance of bringing individuals of different expertise together to work collaboratively towards shared goals:

Figure 1.9 – Anti-type pattern 3

SRE is a discipline that incorporates aspects of software engineering and applies them to infrastructure and operations problems. The main goal of an SRE team is to create scalable and highly reliable software systems. While SRE aligns closely with the DevOps philosophy, merely renaming an operations team to SRE without adopting its principles or practices can be considered an anti-pattern. It is not just about the title but about embracing the methodologies, practices, and culture that both DevOps and SRE advocate for.

Important note

For a more detailed analysis of DevOps anti-types and patterns, please refer to the book Team Topologies by Matthew Skelton and Manuel Pais.

While DevOps itself is not a tool or product, it requires the use of tools to effectively implement its processes and practices. Both open source and commercial tools are available to support the necessary processes for every phase of the DevOps workflow discussed earlier in this chapter (Plan, Code, Build and Test, Release and Deploy, and Operate and Monitor).

Common tools used in the planning phase include Trello, JIRA, Notion, and Asana. According to the latest Stack Overflow Developer Survey, professional developers prefer JIRA (49%), whereas Trello is most used by those learning to code (43%):

Figure 1.10 – Common tools used in the planning phase

During the code and development phase, developers use integrated development environments (IDEs), such as Visual Studio Code, Visual Studio, IntelliJ, Notepad++, and Eclipse, for coding purposes and version control tools, such as Git (self-hosted or cloud-hosted), Apache Subversion (SVN), Perforce, and Mercurial. It is important to note that while this list highlights some of the more common tools, it is by no means exhaustive. There are countless other tools available on the market, each with its unique features and capabilities. According to the 2022 Stack Overflow Developer Survey, professional developers overwhelmingly prefer Git as their version control tool (96%) and Visual Studio Code as their IDE (74%):

Figure 1.11 – Common code and development tools

Important note

The Stack Overflow Developer Survey is an annual survey conducted by Stack Overflow, a popular online community for developers. The survey aims to gather insights into the preferences, opinions, and demographics of the developer community. The 2022 edition can be found here: https://survey.stackoverflow.co/2022.

In the build and test phase, tools such as Jenkins (an open source automation server), Travis CI, and Circle CI are widely used for continuous integration and to build and test automation. According to a recent survey by Digital.ai, Jenkins is used by 56% of DevOps teams, showing its popularity in the industry. In addition, test tools such as Selenium, Junit (a unit testing tool for Java), Nunit (a unit testing tool for .NET), PHPUnit (a unit testing tool for PHP), and Jmeter (a load testing tool for performance testing) can be integrated with build automation servers to facilitate testing procedures. Container build tools such as Docker Build (a tool for building container images from a Dockerfile), Podman Build (a tool for building and managing containers using Containerfiles and Dockerfiles), Buildah (an open source tool for creating and modifying container images), and Kaniko (a secure container build tool designed for Kubernetes clusters) can also be integrated to streamline container image building.

Figure 1.12 – Common tools used in the build and test phase

During the release and deploy phase, developers use various tools to automate deployments. The following table shows some of the tools used in the release and deploy phase:

Table 1.1 – Tools used in the release and deploy phase

During the operate and monitor phase, several tools can be used. Some are highlighted in the following table:

Table 1.2 – Tools used in the operate and monitor phase

A tool such as Prometheus can be used to instrument application code and generate telemetry data such as metrics, logs, and traces. Prometheus, Grafana, and ELK stack (Elasticsearch, Logstash, or Kibana) can be utilized to monitor the performance and availability of applications and infrastructure, providing insights into potential issues and enabling proactive remediation.

Collaboration and communication tools such as Slack, Microsoft Teams, Azure Boards, and Atlassian Confluence can be used to facilitate communication and collaboration between teams, helping to streamline workflows and improve productivity.

Developers have access to a wide variety of tools for each phase that extends beyond the ones we have mentioned. To understand the abundance of tooling options available, we suggest referring to the cloud-native landscape map provided by the Cloud Native Computing Foundation (CNCF) at https://landscape.cncf.io/. The map (Figure 1.13) is designed to help people navigate the various tools, technologies, and platforms that are available in the cloud-native space. It showcases tooling across several categories, such as application development, continuous integration and delivery, automation, and configuration.

Figure 1.13 – A screenshot of the CNCF landscape map

As teams adopt DevOps practices, they often select multiple tools based on preferences rather than considering overall compatibility with the organization’s DevOps strategy (unfortunately, many organizations do not have a defined strategy for adopting DevOps). As a result, fragmented toolchains can be a common occurrence where different teams and product units use different tools that may not integrate or work well together, hindering the ability to scale software delivery and leading to governance challenges. With multiple tools in use, it can be difficult to establish and enforce governance and compliance policies related to access control and data privacy. To address these challenges, a platform approach to tooling may be preferred.

The platform approach to DevOps tooling

Instead of using multiple disjointed tools for each stage of the DevOps workflow, some organizations opt for a platform strategy that offers a single integrated platform with tools for each phase. This approach can simplify the DevOps tooling landscape, making it easier to manage and reducing the need for manual integration between different tools.

Based on industry reports and surveys, here are five of the most commonly used and highly regarded commercial DevOps platform offerings:

• GitLab: An all-in-one DevOps platform that provides a single application for source code management, continuous integration, testing, and deployment.
• Azure DevOps: A Microsoft cloud-based platform that offers a set of DevOps services for developers to plan, develop, test, and deploy applications.
• GitHub: Another Microsoft cloud-based platform that offers a set of DevOps services for developers to plan, develop, test, and deploy applications.
• Atlassian: Atlassian offers a range of tools for DevOps teams, including Jira for issue tracking, Bitbucket for source code management, and Bamboo for continuous integration and deployment.
• Amazon Web Services (AWSs) DevOps: AWSs offers a suite of tools and services for DevOps, including AWS CodePipeline, AWS CodeCommit, and AWS CodeDeploy.

Two of these platforms are Microsoft offerings that bring the tools needed to implement DevOps processes together in one place: Azure DevOps and GitHub.

An overview of the Azure DevOps platform

Azure DevOps is a Microsoft cloud platform with services that help teams implement DevOps processes. To use it, we need to create an Azure DevOps Organization (Figure 1.14). Within the organization, we can create separate projects for different software projects that we are working on, as shown in Figure 1.14. Within each project, we have access to the services that we can use to implement DevOps processes, and we can organize teams to work on different parts of the project:

Figure 1.14 – Azure DevOps Organization hierarchy

The Azure DevOps platform has five core services. These services are connected to key practices in the development process, such as planning, controlling changes to code, and testing. These are the five core services of Azure DevOps:

• Azure Boards for planning
• Azure Repos for controlling code changes
• Azure Pipelines for continuous integration and delivery
• Azure Artifacts for package management
• Azure Test Plans for exploratory test planning

Figure 1.15 – Azure DevOps core services

Let’s briefly look at these five services, starting with Azure Boards:

• Azure Boards: A tool that helps us to plan, track, and visualize work, similar to JIRA. It can be used with Scrum or Kanban methods and has four different templates from which to choose. It also has interactive boards and reporting tools to help us keep track of our work.
• Azure Repos: A source control management service for managing changes to code. It works with two types of code management: Git and team foundation version control (TFVC). It is also integrated with other services in Azure DevOps for traceability.
• Azure Pipelines: A tool that helps us to automatically build, test, and deploy code. It can be used to implement the process of continuous integration and continuous delivery. It works with many different types of programming languages and platforms, including Python, Java, PHP, Ruby, C#, and Go. We can also use it to deploy your code to various types of targets, including on-premises servers or cloud services.
• Azure Artifacts: A tool that helps us to store, manage, and organize software packages. We can choose and control who we want to share packages with. It allows us to download packages from upstream sources. It works with different types of packages, such as NuGet, NPM, Maven, Universal, and Python.
• Azure Test Plans: A cloud-hosted test management solution that we can use to plan and track the results of different types of tests. We can use it to plan and track manual tests, user acceptance tests, exploratory tests, and even automated tests. We can use any supported browser to access the tool and run manual tests through an easy-to-use web portal. It supports end-to-end traceability for tracking the progress and quality of our requirements and builds and provides us with data and reports to improve our testing processes.

One good thing about the Azure DevOps platform is that we’re not forced to use its services. We can choose which services we want to use for a software project and turn off the ones we don’t need (Figure 1.16).

Figure 1.16 – Enable/Disable Azure DevOps services

An overview of the GitHub platform

The GitHub platform provides a variety of product options to accommodate teams and organizations of varying sizes. The options include the following:

• GitHub Free: This is a free, basic version that is good for small personal projects or open source projects.
• GitHub Pro: This is a paid version that has extra features such as advanced protection capabilities, protected branches, and code owners. It’s good for developers who need more advanced tools.
• GitHub Team: This version includes all of the features of GitHub Pro and has team management tools. It’s good for teams that need to collaborate on projects. If your organization has 11 or fewer developers, consider GitHub Team.
• GitHub Enterprise: This version is for large organizations that need even advanced features such as SAML single sign-on (SSO), data residency compliance, and advanced security capabilities. It’s good for large organizations that need to follow specific security and regulatory requirements. Organizations with 12 or more developers typically benefit the most from GitHub Enterprise. The Enterprise version also offers two options: Enterprise server, which is hosted on customer-managed infrastructure, and Enterprise cloud, which is cloud-hosted.

Figure 1.17 – GitHub platform product options

Throughout the remainder of this book, our focus will be on the GitHub Enterprise Cloud product offering. For us to use GitHub Enterprise Cloud, we need to create a GitHub Organization (Figure 1.18). An organization is a shared, private GitHub account where enterprise members can collaborate across many projects at once. Within the organization, we can create repositories, which are like projects in Azure DevOps. It is a good idea to create a separate repository for each project that the organization is working on.

Figure 1.18 – GitHub Organization hierarchy

A company can have multiple GitHub organizations. To simplify visibility, management, and billing, it is recommended to create an enterprise account to manage all organizations that belong to your company (Figure 1.19). Creating an enterprise account is optional, but it is free and will not add any additional charges for GitHub Enterprise Cloud customers. Even if a company only has one organization, it is still beneficial to create an enterprise account. With an enterprise account, we can manage and enforce policies for all the organizations owned by our company. We can even choose policies that we want to enforce at the enterprise level while allowing organization owners to configure other policies at the organization level.

Figure 1.19 – GitHub Enterprise Account

The GitHub Enterprise Cloud platform offers a range of services that we can use for different stages of the code-to-cloud process. These services include the following:

• Projects for planning, organizing, collaborating, and tracking software development projects.
• Codespaces for writing code in a cloud-based development environment.
• Copilot for machine learning-assisted code writing.
• Repos for managing private and public code repositories.
• Actions for automating building, testing, and deployment of code.
• Packages for sharing and discovering reusable code packages.
• Security for scanning and detecting security issues in code repositories.

The following image shows the layout of the GitHub services:

Figure 1.20 – GitHub services

Let’s briefly look at these five services, starting with GitHub Projects:

• GitHub Projects: A tool that we can use to plan, organize, and keep track of software projects. We can use it to assign tasks, collaborate with others, and add extra information to keep track of progress. It also has the capability to report on completed and outstanding work.
• Codespaces: This offers a convenient cloud-based development environment where developers can run, test, debug, and push code without the need for local machine setup. Upon creating a codespace, developers are automatically provided with an already configured system that includes SDKs and runtime for various languages such as Python, Node, Docker, Java, Rust, Go, and C++. The default image can be fully customized to suit individual or team needs, allowing for a faster setup time for each repository.
• GitHub Copilot: An AI pair programmer tool powered by OpenAI Codex, a machine learning model developed by OpenAI (a popular AI research and deployment company). Copilot provides code suggestions as developers write code in their IDEs. It can also interpret natural language comments and turn them into code. It supports multiple programming languages as it is trained on all languages that appear in public repositories. Copilot can be used as an extension in supported IDEs, such as Visual Studio Code, Visual Studio, Neovim, and the JetBrains suite of IDEs.
• GitHub Repos: A source control management service for managing changes to code. Unlike Azure DevOps, it only supports Git, which is a distributed source control. It is also integrated with other services in GitHub for traceability.
• GitHub Actions: A tool that helps us to automatically build, test, and deploy code. It can be used to implement the process of continuous integration and continuous delivery. It works with many different types of programming languages and platforms, including Python, Java, PHP, Ruby, C#, and Go. We can also use it to deploy code to various types of targets, including on-premises servers or cloud services.
• GitHub Packages: A tool that helps us to store, manage, and organize software packages. We can choose and control who we want to share packages with. It allows us to download packages from upstream sources. It works with different types of packages, such as NuGet, NPM, Maven, Universal, and Python.
• GitHub Advanced Security: This provides a range of tools to secure code in our repositories. It scans for vulnerable dependencies and allows us to automatically raise pull requests to fix them. It detects security vulnerabilities and coding errors in new or modified code. It can also identify any tokens or credentials accidentally committed to a repository. We will discuss this service in detail in the later chapters of this book.

Let’s have a quick look at another DevOps platform: GitLab.

An overview of the GitLab platform

GitLab is a web-based Git repository management tool that provides an end-to-end DevOps solution. Similar to other DevOps platforms, GitLab also has core services that support various stages of the DevOps workflow. These services are the following:

• GitLab Issues: It is an Agile project management tool that helps teams to plan and organize their work using either Scrum or Kanban methodologies. With GitLab Boards, teams can easily track their progress, visualize their work, and collaborate with team members.
• GitLab Repository: GitLab is primarily known for its version control system. It provides a centralized platform for teams to store, manage, and collaborate on their codebase using Git. Teams can use GitLab Repository with either Git or Mercurial, and they can easily import their codebase from other repositories.
• GitLab CI/CD: GitLab’s CI/CD tool allows teams to automate their software delivery processes. GitLab CI/CD enables teams to build, test, and deploy their applications across various environments in a secure and efficient manner.
• GitLab Container Registry: GitLab Container Registry is a built-in container registry that enables teams to store, manage, and deploy their Docker images. Teams can use GitLab Container Registry to create and manage their images and then deploy them to their preferred platform.
• GitLab Monitor: GitLab Monitor is a monitoring tool that provides real-time visibility into the performance of applications and infrastructure. Teams can use GitLab Monitor to monitor the health of their applications and infrastructure, detect issues, and resolve them quickly.

GitLab is also highly configurable and customizable. Teams can easily customize the platform to fit their needs and preferences. GitLab supports various integrations and has a vast ecosystem of third-party extensions and plugins that teams can use to extend their functionalities.

Azure services for the DevOps workflow

Microsoft Azure offers a wide range of tools and services that can integrate well into a DevOps workflow. A broad range of tools and services for secret management, configuration management, load testing, chaos engineering, and app hosting/deployment, as well as comprehensive monitoring and observability capabilities.

Figure 1.21 – Azure Cloud-native services for DevOps

Figure 1.21 highlights some of the tools that can be used in the different stages of the DevOps workflow. Let us review some of these services and how they fit in:

We have various services to host our applications:

1. Build phase:
   • Azure Key Vault: This offers secure secret management, allowing developers to store and retrieve sensitive information such as API keys, passwords, and certificates.
   • Azure App Configuration: This enables centralized configuration management, providing a way to store and retrieve application settings across multiple environments.
2. Test phase:
   • Azure Load Testing: This allows for the stress testing and performance testing of applications by simulating user traffic and analyzing system behavior under load.
   • Azure Chaos Studio: This facilitates chaos engineering experiments by introducing controlled disruptions and failures to test system resiliency.
3. Release phase: Azure offers several computing options for app hosting and deployment:
   • Virtual machines (VMs) and VM scale sets: These offer flexibility to deploy and manage virtual machines for hosting applications.
   • App Services: This provides a platform to host web and API applications without worrying about infrastructure management.
   • Function Apps: This enables the development of serverless functions to execute code on demand.
   • Container Services: This supports containerized application deployments with options such as Azure Container Instances for lightweight workloads or Azure Kubernetes Service for orchestrating and scaling containerized applications.
4. Operate and monitor phases:
   • Azure Monitor: This offers comprehensive monitoring and diagnostics for applications and infrastructure, allowing teams to gain insights into system performance and health.
   • Application Insights: This provides real-time application performance monitoring and logging, allowing developers to detect and diagnose issues quickly.
   • Managed Grafana for observability: This integrates Grafana, a popular open source observability platform, with Azure services, enabling advanced data visualization and analysis for monitoring and troubleshooting.

Keep in mind that the examples mentioned here are just a few, and we will encounter more services as we progress. Throughout this book, we will explore various Azure services that support DevOps practices and enhance the software development process, particularly those related to security use cases.

For now, just note that DevOps and cloud computing go hand in hand, as both are designed to enable faster software development and deployment. The cloud provides a scalable and flexible infrastructure that can support the demands of modern software development and services that enhance the process, and DevOps provides a framework for efficiently managing and deploying software in the cloud.

Now that we have explored the fundamental concepts of Agile, DevOps, and cloud computing, let us examine how these three elements come together to enable modern software development practices.

Adopting a DevOps approach does not yield benefits in isolation but rather in conjunction with other concepts, such as Agile planning and cloud computing. Agile is a way of managing a project that focuses on being flexible and responsive to change. cloud computing refers to using web-based computing services instead of physical servers and software. Together, Agile, DevOps, and Cloud can help organizations work more effectively and efficiently.

Some organizations may use only one or two of these concepts, but the best results come from combining all three. It is possible to adopt an Agile approach to software development without practicing DevOps; it is also possible to implement DevOps practices but not with cloud computing, and it is, sadly, common for many organizations to adopt cloud computing without implementing DevOps practices. For cloud-native applications and new software, the synergy of all three—Agile, DevOps, and Cloud—often yields the best outcomes, as illustrated in the following diagram:

Figure 1.22 – Agile, DevOps, and Cloud

We put it this way: Agile is what we should be doing; DevOps is how we should be doing it; cloud computing is where we should be doing it.

However, it is essential to recognize that there are exceptions to this general rule. Combining Agile, DevOps, and cloud computing for certain applications can present a different set of challenges and dynamics. While the integration of these three elements can be highly beneficial for many applications, it is not a guaranteed formula for success in every scenario. As the saying goes, not everything that glitters is gold, and not every combination of DevOps and cloud will yield golden results.

Let us move on from our discussions for now. In the next sections, we will set up the required cloud accounts necessary to follow along with the hands-on exercises covered in the rest of this book.

Let’s start with creating a subscription:

1. Open a web browser and go to https://signup.azure.com.
2. Click on Sign in.
3. Enter your profile information, verify your identity, and agree to the terms and conditions.
4. Click on Next and provide your credit card information (note that your credit card will not be charged until you switch your subscription from the free trial to a paid subscription).
5. Click on Sign up.

Next, we’ll create a new organization:

1. Open a web browser and go to https://portal.azure.com (the Azure portal).
2. In the Azure portal, in the search area at the top, search for and select Azure DevOps.
3. Click on the My Azure DevOps Organizations link.
4. In the open window, configure the following:
   • Name: Enter a name for your new Azure DevOps Organization
   • Project Location: Select a location close to you

Next, let’s configure billing for our new organization:

1. In the Azure DevOps console, click on Organization Settings in the lower-left corner.
2. Click on Billing, then click on Set up Billing.
3. Select your Azure subscription, then click on Save.

First, let’s create a GitHub Enterprise Cloud organization:

1. To create a GitHub Organization, go to https://github.com/pricing, click on Start a free trial under Enterprise, and then choose Enterprise Cloud:

Figure 1.23 – Creating Enterprise Cloud Organizations

2. Sign in to GitHub or create an account.
3. After signing in, you will be directed to the Set up your Enterprise trial page. Fill in the details required and click the Create your enterprise button.

Well done! You have now completed the setup of the cloud accounts required for the hands-on exercises in the upcoming chapters.

This chapter provided an overview of DevOps—a modern application development and delivery approach that helps organizations to release quality software more quickly into production with fewer defects! We covered the five core practices of DevOps, the different stages in a DevOps workflow, and the importance of a collaborative culture in achieving success with DevOps. We also highlighted the DevOps anti-types and anti-patterns to avoid and introduced popular DevOps platforms, such as Azure DevOps, GitHub Actions, and GitLab. The knowledge gained in this chapter has equipped you with strong foundational knowledge that is needed to understand the discussions presented throughout the remainder of this book.

In the next chapter, we will delve into the security challenges of DevOps, exploring the potential risks and ways organizations can start to address them. See you there!

To learn more about the topics that were covered in this chapter, take a look at the following resources:

• Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations by Nicole Forsgren, Jez Humble, and Gene Kim.
• The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win by Gene Kim, Kevin Behr, and George Spafford.