The rise of DevOps
The rise of cloud computing since mid-2000 has been spectacular. Hardly a month goes by without the three hyperscalers (Amazon Web Services (AWS), Azure, and Google Cloud) announcing the opening of a new data center region. Cloud computing—in particular, public cloud—offers an incredible array of technologies at a nearly infinite scale. This has led to a new way of deploying and operating IT. DevOps combines two areas that were traditionally distinct phases in software development—development and operations. DevOps aims to deliver software at a much faster pace than the traditional Waterfall model was able to. By combining historically distinct phases and teams, DevOps methodology can deliver software much more rapidly and with higher quality than traditional software methodology.
One key aspect of DevOps is automation. Combining several separate tools into a pipeline, we can deliver software from development to production with minimal human intervention. This concept of continuous integration and continuous delivery, usually referred to as CI/CD, integrates managing source code, provisioning the IT infrastructure, compiling (if necessary), packaging, testing, and deploying into a pipeline. A CI/CD pipeline requires automation at every step to execute efficiently.
Infrastructure as Code
Automating the provisioning of the IT infrastructure is a key component of a CI/CD pipeline and is known as IaC. In traditional on-prem computing, servers and networking infrastructure provision was a long-drawn and manual process. It started with ordering IT hardware, the physical setup of the hardware, and configuration, such as installing the operating system and configuring the network infrastructure. This process would often take weeks or months. Virtualization somewhat helped the process, but the provisioning of the infrastructure would generally still fall onto a separate team before developers could start deploying the fruit of their labor into a test environment, much less a production one.
The rise of cloud computing shortened this process from months to days and even hours. Infrastructure can now be provisioned through a graphical user interface (GUI) known as the web console. Initially, this was a straightforward process as the number of services and configuration options were manageable. However, as more services became available and the configuration options increased exponentially, provisioning networks, servers, databases, and other managed services became tedious and error-prone.
A key objective in DevOps is to have essentially identical environments. That is, the development, test, and production environments should be the same except for some minor configuration changes; for example, the database in development and production should be identical except for the number of CPUs and the size of memory.
However, configuring complex environments using the web console is tedious and error prone. Using what is sometimes derided as ClickOps, the provisioning of even a medium-complex environment requires hundreds, if not thousands, of user interactions or clicks.
Using a CLI can help with configuring multiple environments. One can develop scripts that combine multiple CLI commands into a single executable. However, managing and making changes is next to impossible using a CLI.
Enter IaC. IaC is the provisioning of infrastructure using code rather than a web console or a CLI. For example, you can write code to achieve the same step instead of using the web console and going through several manual steps to provision a server. For example, the main.tf file featured in this chapter shows the configuration of a server with the Debian operating system. The server, called cloudshell, is an e2-micro instance placed in the us-central1-a region.
Once you have the code, you can reuse it repeatedly. That is, you can deploy many servers with minimal or no change. Since, it is regular code, you can use version control and source code revision systems to manage your code. This facilitates effective teamwork.
Furthermore, IaC can be integrated into a CI/CD pipeline, including testing and validation. That is, you can validate and test the provisioning of servers before they are deployed. Thus, you can provision complex infrastructure involving hundreds of servers, complex networking, and multiple services within minutes rather than days or weeks. Ultimately, this makes the software development release process faster and more secure, which is precisely the objective of DevOps.
A CI/CD pipeline includes many steps besides IaC. One of the steps is configuration management, which includes configuring servers such as installing updates, libraries, and code. Ansible, Puppet, and Chef are common configuration management tools. Some overlap exists between configuration management tools and IaC tools. Configuration management tools can provision some infrastructure, while IaC tools perform some configuration tasks. However, infrastructure provisioning and configuration management should generally be considered two separate steps better served by different tools.
Terraform
Terraform has become the most popular IaC tool in recent years. While each cloud platform has its proprietary IaC tool (AWS has CloudFormation, Azure has Azure Resource Manager, and Google Cloud has Deployment Manager), Terraform is unique in that it is platform agnostic. You can use Terraform to provision infrastructure in any cloud provider and for many other platforms such as vSphere and Google Workspace.
As you will see, Terraform is easy to learn and provides a straightforward workflow that can be easily incorporated into any CI tool. Terraform is open source and has developed an active community with its own ecosystem. Community members have developed tools that help you write Terraform code more effectively; we cover some of them in this book.
