This chapter and the remainder of this book require you to have a Google Cloud account. You can use an existing Google Cloud project, but we recommend creating a new clean project for you to follow along. You should also have the Google Cloud command-line interface (CLI) installed on your local PC and be familiar with basic gcloud commands. Please see https://cloud.google.com/sdk/docs/install for detailed instructions on how to download the CLI.

Of course, we are using Terraform. Terraform is available on all common operating systems and is easy to install. You can download the version for your operating system at https://www.terraform.io/downloads. HashiCorp is constantly improving the tool by providing updates and upgrades. For the writing of this book, we are using v1.3.3. The code should work with any version greater than v.1.3.0. However, we suggest you download this particular version if you run into any issues.

The source code for this chapter and all other chapters is available at https://github.com/PacktPublishing/Terraform-for-Google-Cloud-Essential-Guide.

We recommend that you download the code, enabling you to follow along. We organized the code into analogous chapters and sections and indicated the appropriate subdirectories.

The rise of cloud computing since mid-2000 has been spectacular. Hardly a month goes by without the three hyperscalers (Amazon Web Services (AWS), Azure, and Google Cloud) announcing the opening of a new data center region. Cloud computing—in particular, public cloud—offers an incredible array of technologies at a nearly infinite scale. This has led to a new way of deploying and operating IT. DevOps combines two areas that were traditionally distinct phases in software development—development and operations. DevOps aims to deliver software at a much faster pace than the traditional Waterfall model was able to. By combining historically distinct phases and teams, DevOps methodology can deliver software much more rapidly and with higher quality than traditional software methodology.

One key aspect of DevOps is automation. Combining several separate tools into a pipeline, we can deliver software from development to production with minimal human intervention. This concept of continuous integration and continuous delivery, usually referred to as CI/CD, integrates managing source code, provisioning the IT infrastructure, compiling (if necessary), packaging, testing, and deploying into a pipeline. A CI/CD pipeline requires automation at every step to execute efficiently.

Infrastructure as Code

Automating the provisioning of the IT infrastructure is a key component of a CI/CD pipeline and is known as IaC. In traditional on-prem computing, servers and networking infrastructure provision was a long-drawn and manual process. It started with ordering IT hardware, the physical setup of the hardware, and configuration, such as installing the operating system and configuring the network infrastructure. This process would often take weeks or months. Virtualization somewhat helped the process, but the provisioning of the infrastructure would generally still fall onto a separate team before developers could start deploying the fruit of their labor into a test environment, much less a production one.

The rise of cloud computing shortened this process from months to days and even hours. Infrastructure can now be provisioned through a graphical user interface (GUI) known as the web console. Initially, this was a straightforward process as the number of services and configuration options were manageable. However, as more services became available and the configuration options increased exponentially, provisioning networks, servers, databases, and other managed services became tedious and error-prone.

A key objective in DevOps is to have essentially identical environments. That is, the development, test, and production environments should be the same except for some minor configuration changes; for example, the database in development and production should be identical except for the number of CPUs and the size of memory.

However, configuring complex environments using the web console is tedious and error prone. Using what is sometimes derided as ClickOps, the provisioning of even a medium-complex environment requires hundreds, if not thousands, of user interactions or clicks.

Using a CLI can help with configuring multiple environments. One can develop scripts that combine multiple CLI commands into a single executable. However, managing and making changes is next to impossible using a CLI.

Enter IaC. IaC is the provisioning of infrastructure using code rather than a web console or a CLI. For example, you can write code to achieve the same step instead of using the web console and going through several manual steps to provision a server. For example, the main.tf file featured in this chapter shows the configuration of a server with the Debian operating system. The server, called cloudshell, is an e2-micro instance placed in the us-central1-a region.

Once you have the code, you can reuse it repeatedly. That is, you can deploy many servers with minimal or no change. Since, it is regular code, you can use version control and source code revision systems to manage your code. This facilitates effective teamwork.

Furthermore, IaC can be integrated into a CI/CD pipeline, including testing and validation. That is, you can validate and test the provisioning of servers before they are deployed. Thus, you can provision complex infrastructure involving hundreds of servers, complex networking, and multiple services within minutes rather than days or weeks. Ultimately, this makes the software development release process faster and more secure, which is precisely the objective of DevOps.

A CI/CD pipeline includes many steps besides IaC. One of the steps is configuration management, which includes configuring servers such as installing updates, libraries, and code. Ansible, Puppet, and Chef are common configuration management tools. Some overlap exists between configuration management tools and IaC tools. Configuration management tools can provision some infrastructure, while IaC tools perform some configuration tasks. However, infrastructure provisioning and configuration management should generally be considered two separate steps better served by different tools.

Terraform

Terraform has become the most popular IaC tool in recent years. While each cloud platform has its proprietary IaC tool (AWS has CloudFormation, Azure has Azure Resource Manager, and Google Cloud has Deployment Manager), Terraform is unique in that it is platform agnostic. You can use Terraform to provision infrastructure in any cloud provider and for many other platforms such as vSphere and Google Workspace.

As you will see, Terraform is easy to learn and provides a straightforward workflow that can be easily incorporated into any CI tool. Terraform is open source and has developed an active community with its own ecosystem. Community members have developed tools that help you write Terraform code more effectively; we cover some of them in this book.

Note

The code for this section is under chap01/cloudshell in the GitHub repo of this book.

The easiest way to run Terraform in Google Cloud is using Google Cloud Shell, which is a preconfigured development environment accessible through your browser. It comes pre-installed with the latest version of common utilities, including the latest version of Terraform. Furthermore, all authentication is already set up. So, let’s give it a try.

Note

You can check the current version of Terraform using the terraform –version command.

If your project is new, and you have not provisioned a virtual machine (VM) (Compute Engine), run the following gcloud command to enable the compute API:

$ gcloud services enable compute.googleapis.com

Then place the following code in a file called main.tf. This is known as the Terraform configuration. Configurations are written in HashiCorp Configuration Language (HCL). HCL is human readable and is used by several HashiCorp tools.

Note

In this book, we will use the term Terraform language to refer to the language in which the configuration files are written.

This book aims to make you proficient in Terraform to write high-quality, reusable code to provision resources in Google Cloud efficiently and securely.

The file’s actual name is arbitrary, but note that the .tf extension is mandatory. Terraform uses the .tf extension to identify Terraform configuration files.

main.tf

resource "google_compute_instance" "this" {
  name         = "cloudshell"
  machine_type = "e2-small"
  zone         = "us-central1-a"
  boot_disk {
    initialize_params {
      image = "debian-cloud/debian-11"
    }
  }
  network_interface {
    network = "default"
  }
}

Next, run the following two commands:

$ terraform init
$ terraform apply

Terraform will ask you where you want to perform these actions, so type in yes to approve.

Congratulations—you have now provisioned your first server using Terraform! Go to the web console and inspect the compute engine that Terraform has created.

As expected, Terraform created a Debian server named cloudshell of a machine type e2-small in the default network:

Figure 1.1 – Terraform-provisioned server in the web console

Let’s have a more detailed look at how to write Terraform configurations and how Terraform operates.

As mentioned, the Terraform language is based on HCL. The most basic construct is a block, defined as a “container for other content” (https://developer.hashicorp.com/terraform/language/syntax/configuration#blocks). For example, the preceding code block is of the type resource. A resource block has two labels. The first label consists of the provider and resource name. For example, the resource label for a Google compute engine is google_compute_instance, whereas the label for an AWS E2 instance is aws_instance. The second label in a resource block is the ID, the name you give so that Terraform can uniquely identify each resource. Please note that no two resources can have the same name. So, if you want to provision two VMs, you define two resource blocks, each of type google_compute_instance, and each with a unique name.

The body of the block is nested within {}. Blocks can have nested blocks. In the preceding code example, there are two nested blocks, boot_disk (which in itself has a nested block called initialize_params) and network_interface.

While the syntax for Terraform is the same, each cloud provider has its unique resources and data source definitions. The Terraform documentation for Google Cloud is at https://registry.terraform.io/providers/hashicorp/google/latest/docs. You see that all resources and data sources start with the google keyword. In general, it is followed by the name of the equivalent gcloud command. Thus, the resource to create a compute instance is named google_compute_instance, whereas the resource to provision a SQL database instance is named google_sql_database_instance.

Looking at the documentation, you can see the current Google services that are currently supported. More are added as Google Cloud introduces new services. We recommend you bookmark the documentation as you have to refer to it frequently.

In the first part of this book, we use only a few Google Cloud resources as we want to concentrate on the language. We design complex architectures using additional Google Cloud services in the second part. But first, let’s look at the steps involved in using Terraform.