HCL versus JSON

It’s good to have some basic background on the three coding formats supported by Packer templates. JSON is a descriptive language that uses blocks to declare a data structure. A JSON document may use an optional schema that is a secondary JSON document that lists the structure for the writer to follow. Since version 1.7, Packer actually supports two versions of JSON, so it’s important to know how to identify them by file extension when coming across older templates. Legacy templates end in just .json, whereas new templates end in .pkr.json, and both options use different schemas or styles. HCL is HashiCorp’s own syntax, which has a few more features than JSON but also a few limitations:

	JSON	HCL (version 2)
Pros	Widely used across the industry Supports schemas IDE support	Comment support Complex constructs, for loops, and here documents Helpful parameters IDE support
Cons	No comments Strict format Lack of constructs: for loops and here files, also known as heredocs	No schema support

Table 1.1 – Comparison of JSON and HCL code for Packer

The good news is, Packer supports both HCL and JSON and also has a helpful tool to convert an existing JSON template into an HCL template automatically. HCL may support schemas in the future, but currently, its open format features also help make it more flexible and easier to read than JSON in some cases. Let’s start with some examples of both JSON and HCL2 Packer templates. Note there are actually two versions of the JSON schema supported by Packer. The one you use must be reflected in the file extension when you save your template. Legacy JSON templates just end in *.json and are supported for existing templates in the community. Newer JSON templates should be written in HCL2.

Example legacy JSON

The following sample is an excerpt from a legacy JSON template used to build an image on VMware. You may encounter these in older examples and Packer still supports them for backward compatibility. Note that some JSON strings contain Go-style templating, indicated by double braces, {{ }}. Adding comments is not an option in JSON, so it is difficult to document your code. This code starts with a CentOS 7.8 image, boots it on VMware as specified by a builder, and then uses a provisioner to upload a script and another provisioner to run that script.

JSON schemas provide a way to describe the possible options for a desired JSON document, and can help guide a coder with suggestions, auto-completion, and type checking while building a template. Schemas can also generate WYSIWYG editors, which allow automatic menus and designers for those who don’t want to write code manually. Partial community schemas for Packer templates have been written by the author and are available at https://github.com/jboero/hashicorp-schemas/blob/master/JSON/packer/1.5/template.json. These schemas are community-driven, not created by HashiCorp engineers. Note that these template samples won’t build for you unless you specify a compatible base image. We will actually cover a practical example in the next chapter. A sample template in HCL2 is given in the following code block. We will break down this template line by line in the coming chapters. Optional variables can be declared to help make templates reusable. These definitions look like this and let you define whatever variables you like. Here, there are three variables with default values declared that will be used in builder declarations:

variable "base_url" {
  type        = string
  default     = "https://my-source/image.iso"
  description = "URL for our base image"
  sensitive   = false
}

Variables in Packer’s HCL2 format also offer optional validation blocks. This is helpful for limiting what you can assign to the variable. For example, the base_url variable in the preceding example is a URL and we want to restrict it to take only values starting with https, we can specify this using this validation block:

validation {
  condition     = substr(var.base_url, 0, 5) == "https"
  error_message = "URLs must start with https"
}

There are many variables that come built into Packer for each build or source. These give access to dynamic values, such as the unique identifier for the build, name, and ID of the build resource. This is helpful when you want to inject aspects about the build itself into actions or provisioners performed in each environment. For example, if you want to save the Packer build UUID into the image via a file such as /etc/packerbuild, you can reference the build.PackerRunUUID variable. A list of the build and source variables can be found in Packer’s contextual variable documentation: https://developer.hashicorp.com/packer/docs/templates/hcl_templates/contextual-variables.

Builders are plugins used to declare an environment for image building, such as VMware, VirtualBox, QEMU, and Docker. As of Packer version 1.7, templates declare an instance of a builder as a source. In this sample, we declare one builder of the VMWare ISO type with minimal settings to connect our VM. Notice the previous variables are inserted into strings using the {{ }} templating syntax. HCL also supports direct variable usage without strings. A builder says nothing about how your image should be customized. It only tells Packer what kind of environment to run provisioners on to customize your image. Take this example:

source "vsphere-iso" "example" {
  iso_url = var.base_url
  iso_checksum = var.base_checksum
  ssh_username = "packer"
  ssh_password = "packer"
  shutdown_command = "shutdown -P now"
  boot_command = [
    "<esc><wait>",
    "vmlinuz initrd=initrd.img ",
    "<enter>"]
  boot_key_interval = "1ms"
  boot_wait = "1s"
  cpus = 8
  memory = 8192
  disk_size = 4000
}

Provisioners are the magic of Packer. These are customizations, resources, or scripts that should be run on all of the builders to preconfigure everything you expect in the image. Once all of the provisioners are finished, Packer saves the image as configured in the builder. Here, there are two provisioners. The first is a script called install.sh, which we upload into the builder from a local directory, ./http/install.sh. Then, the second provisioner is a shell command to run that script:

provisioner "file" {
  destination = "/tmp/install.sh"
  Source = "./http/install.sh"
  direction = "upload"
}
provisioner "shell" {
  inline = ["sudo bash –x /tmp/install.sh"]
}

Packer can be used to build or simply validate this JSON document as a valid template. Note that JSON templates require a root document. Everything is nested within a single set of braces, also known as a code block. This differs from HCL, which requires no root document or block.

Example PKR.JSON

When Packer added HCL2 support, it restructured how templates are structured. There is an additional JSON option that mirrors this HCL2 format. Builders are instead defined as sources and then a build job lists which sources you would like to include in the build. It may be a little confusing if you are used to legacy JSON support. Packer will select whether your JSON file uses the legacy or new schema by its file extension. For example, template.json uses the legacy schema, as used in the preceding example, whereas template.pkr.json would tell Packer to use the new schema of sources. HCL2 is still the recommended way to build new templates, though JSON support still offers some nice automation options for IDEs and UI wizards, which we’ll discuss in Chapter 2, Creating Your First Template. The equivalent example in pkr.json format is listed in the book’s GitHub repo: https://github.com/PacktPublishing/HashiCorp-Packer-in-Production/blob/main/Chapter01/Sample.pkr.json.

Example HCL

Here, I have taken the previous legacy JSON template and migrated it to HCL2 via Packer’s built-in packer hcl2_upgrade [template.json] command. I have also added some comments to explain what’s happening. HCL supports three comment types: //, /*, and #. I’ve included examples of all of these types in the following snippet, but it’s best to choose one standard and be consistent. HCL has no root object requirement but the structure varies a bit from the JSON version. HCL also supports here docs, also known as here documents, which can help you embed files such as our provisioner script directly into the template. These are often indicated by an <<EOF flag or a similar delimiter. The fully converted template with additional comments added manually is shown here. HCL2 can look quite a bit different than JSON. Variables are declared one at a time like in the following example:

variable "checksum" {
 type    = string
 default = "087a5743dc6fd6...60d75440eb7be14"
}

In addition, each builder is declared separately as a source. Then, a build job lists the sources and provisioners desired:

build {
  sources = ["source.vmware-iso.autogenerated_1"]
  provisioner "file" {
    destination = "/tmp/install.sh"
    direction   = "upload"
    source      = "./http/install.sh"
  }
  provisioner "shell" {
    inline = ["sudo bash -x /tmp/install.sh"]
  }
}

This HCL2 template provides the same details as the JSON version earlier. It has been automatically converted by Packer and commented to provide more detail. In the next chapter, we will break down every line of this template to explain what each value means in detail.