Playbook parsing

The whole purpose of an inventory source is to have systems to manipulate. The manipulation comes from playbooks (or, in the case of Ansible ad hoc execution, simple single-task plays). You should already have a basic understanding of playbook construction, so we won't spend a lot of time covering that; however, we will delve into some specifics of how a playbook is parsed. Specifically, we will cover the following:

• The order of operations
• Relative path assumptions
• Play behavior keys
• The host selection for plays and tasks
• Play and task names

The order of operations

Ansible is designed to be as easy as possible for humans to understand. The developers strive to strike the best balance of human comprehension and machine efficiency. To that end, nearly everything in Ansible can be assumed to be executed in a top-to-bottom order; that is, the operation listed at the top of a file will be accomplished before the operation listed at the bottom of a file. Having said that, there are a few caveats and even a few ways to influence the order of operations.

A playbook only has two main operations it can accomplish. It can either run a play, or it can include another playbook from somewhere on the filesystem. The order in which these are accomplished is simply the order in which they appear in the playbook file, from top to bottom. It is important to note that while the operations are executed in order, the entire playbook and any included playbooks are completely parsed before any executions. This means that any included playbook file has to exist at the time of the playbook parsing – they cannot be generated in an earlier operation. This is specific to playbook inclusions but not necessarily to task inclusions that might appear within a play, which will be covered in a later chapter.

Within a play, there are a few more operations. While a playbook is strictly ordered from top to bottom, a play has a more nuanced order of operations. Here is a list of the possible operations and the order in which they will occur:

• Variable loading
• Fact gathering
• The pre_tasks execution
• Handlers notified from the pre_tasks execution
• The roles execution
• The tasks execution
• Handlers notified from the roles or tasks execution
• The post_tasks execution
• Handlers notified from the post_tasks execution

The following is an example play with most of these operations shown:

--- 
- hosts: localhost 
  gather_facts: false 
 
  vars: 
    - a_var: derp 
 
  pre_tasks: 
    - name: pretask 
      debug: 
        msg: "a pre task" 
      changed_when: true 
      notify: say hi 
 
  roles: 
    - role: simple 
      derp: newval 
 
  tasks: 
    - name: task 
      debug: 
        msg: "a task" 
      changed_when: true 
      notify: say hi
 
  post_tasks: 
    - name: posttask 
      debug: 
        msg: "a post task" 
      changed_when: true 
      notify: say hi 
  handlers:
    - name: say hi
      debug:
        msg: hi

Regardless of the order in which these blocks are listed in a play, the order detailed in the previous code block is the order in which they will be processed. Handlers (that is, the tasks that can be triggered by other tasks that result in a change) are a special case. There is a utility module, ansible.builtin.meta, that can be used to trigger handler processing at a specific point:

- ansible.builtin.meta: flush_handlers 

This will instruct Ansible to process any pending handlers at that point before continuing with the next task or next block of actions within a play. Understanding the order and being able to influence the order with flush_handlers is another key skill to have when there is a need to orchestrate complicated actions; for instance, where things such as service restarts are very sensitive to order. Consider the initial rollout of a service.

The play will have tasks that modify config files and indicate that the service should be restarted when these files change. The play will also indicate that the service should be running. The first time this play happens, the config file will change, and the service will change from not running to running. Then, the handlers will trigger, which will cause the service to restart immediately. This can be disruptive to any consumers of the service. It is better to flush the handlers before a final task to ensure the service is running. This way, the restart will happen before the initial start, so the service will start up once and stay up.

Relative path assumptions

When Ansible parses a playbook, there are certain assumptions that can be made about the relative paths of items referenced by the statements in a playbook. In most cases, paths for things such as variable files to include, task files to include, playbook files to include, files to copy, templates to render, and scripts to execute are all relative to the directory where the file that is referencing them resides. Let's explore this with an example playbook and directory listing to demonstrate where the files are:

• The directory structure is as follows:

  . 
  ├── a_vars_file.yaml 
  ├── mastery-hosts 
  ├── relative.yaml 
  └── tasks 
      ├── a.yaml 
      └── b.yaml 

• The content of a_vars_file.yaml is as follows:

  --- 
  something: "better than nothing" 

• The content of relative.yaml is as follows:

  --- 
  - name: relative path play 
    hosts: localhost 
    gather_facts: false 
    
    vars_files: 
      - a_vars_file.yaml
   
    tasks: 
      - name: who am I 
        ansible.builtin.debug: 
          msg: "I am mastery task" 
      - name: var from file 
        ansible.builtin.debug:         
          var: something 
   
      - ansible.builtin.include: tasks/a.yaml 

• The content of tasks/a.yaml is as follows:

  --- 
  - name: where am I 
    ansible.builtin.debug: 
      msg: "I am task a" 
   
  - ansible.builtin.include: b.yaml 

• The content of tasks/b.yaml is as follows:

  ---
  - name: who am I
    ansible.builtin.debug:
      msg: "I am task b" 

The execution of the playbook is performed with the following command:

ansible-playbook -i mastery-hosts -c local relative.yaml

The output should be similar to Figure 1.9:

Figure 1.9 – The expected output from running a playbook utilizing relative paths

Here, we can clearly see the relative references to the paths and how they are relative to the file referencing them. When using roles, there are some additional relative path assumptions; however, we'll cover that, in detail, in a later chapter.

Play behavior directives

When Ansible parses a play, there are a few directives it looks for in order to define various behaviors for a play. These directives are written at the same level as the hosts: directive. Here is a list of descriptions for some of the more frequently used keys that can be defined in this section of the playbook:

• any_errors_fatal: This Boolean directive is used to instruct Ansible to treat any failure as a fatal error to prevent any further tasks from being attempted. This changes the default, where Ansible will continue until all the tasks have been completed or all the hosts have failed.
• connection: This string directive defines which connection system to use for a given play. A common choice to make here is local, which instructs Ansible to do all the operations locally but with the context of the system from the inventory.
• collections: This is a list of the collection namespaces used within the play to search for modules, plugins, and roles, and it can be used to prevent the need to enter Fully Qualified Collection Names (FQCNs) – we will learn more about this in Chapter 2, Migrating from Earlier Ansible Versions. Note that this value does not get inherited by role tasks, so you must set it separately in each role in the meta/main.yml file.
• gather_facts: This Boolean directive controls whether or not Ansible will perform the fact-gathering phase of the operation, where a special task will run on a host to uncover various facts about the system. Skipping fact gathering – when you are sure that you do not require any of the discovered data – can be a significant time-saver in a large environment.
• Max_fail_percentage: This number directive is similar to any_errors_fatal, but it is more fine-grained. It allows you to define what percentage of your hosts can fail before the whole operation is halted.
• no_log: This is a Boolean to control whether or not Ansible will log (to the screen and/or a configured log file) the command given or the results received from a task. This is important if your task or return deals with secrets. This key can also be applied to a task directly.
• port: This is a number directive to define what SSH port (or any other remote connection plugin) you should use to connect unless this is already configured in the inventory data.
• remote_user: This is a string directive that defines which user to log in with on the remote system. The default setting is to connect as the same user that ansible-playbook was started with.
• serial: This directive takes a number and controls how many systems Ansible will execute a task on before moving to the next task in a play. This is a drastic change from the normal order of operations, where a task is executed across every system in a play before moving to the next. This is very useful in rolling update scenarios, which we will discuss in later chapters.
• become: This is a Boolean directive that is used to configure whether privilege escalation (sudo or something else) should be used on the remote host to execute tasks. This key can also be defined at a task level. Related directives include become_user, become_method, and become_flags. These can be used to configure how the escalation will occur.
• strategy: This directive sets the execution strategy to be used for the play.

Many of these keys will be used in the example playbooks throughout this book.

For a full list of available play directives, please refer to the online documentation at https://docs.ansible.com/ansible/latest/reference_appendices/playbooks_keywords.html#play.

Execution strategies

With the release of Ansible 2.0, a new way to control play execution behavior was introduced: strategy. A strategy defines how Ansible coordinates each task across the set of hosts. Each strategy is a plugin, and three strategies come with Ansible: linear, debug, and free. The linear strategy, which is the default strategy, is how Ansible has always behaved. As a play is executed, all the hosts for a given play execute the first task.

Once they are all complete, Ansible moves to the next task. The serial directive can create batches of hosts to operate in this way, but the base strategy remains the same. All the targets for a given batch must complete a task before the next task is executed. The debug strategy makes use of the same linear mode of execution described earlier, except that here, tasks are run in an interactive debugging session rather than running to completion without any user intervention. This is especially valuable during the testing and development of complex and/or long-running automation code where you need to analyze the behavior of the Ansible code as it runs, rather than simply running it and hoping for the best!

The free strategy breaks from this traditional linear behavior. When using the free strategy, as soon as a host completes a task, Ansible will execute the next task for that host, without waiting for any other hosts to finish.

This will happen for every host in the set and for every task in the play. Each host will complete the tasks as fast as they can, thus minimizing the execution time of each specific host. While most playbooks will use the default linear strategy, there are situations where the free strategy would be advantageous; for example, when upgrading a service across a large set of hosts. If the play requires numerous tasks to perform the upgrade, which starts with shutting down the service, then it would be more important for each host to suffer as little downtime as possible.

Allowing each host to independently move through the play as fast as it can will ensure that each host is only down for as long as necessary. Without using the free strategy, the entire set will be down for as long as the slowest host in the set takes to complete the tasks.

As the free strategy does not coordinate task completion across hosts, it is not possible to depend on the data that is generated during a task on one host to be available for use in a later task on a different host. There is no guarantee that the first host will have completed the task that generates the data.

Execution strategies are implemented as a plugin and, as such, custom strategies can be developed to extend Ansible behavior by anyone who wishes to contribute to the project.

The host selection for plays and tasks

The first thing that most plays define (after a name, of course) is a host pattern for the play. This is the pattern used to select hosts out of the inventory object to run the tasks on. Generally, this is straightforward; a host pattern contains one or more blocks indicating a host, group, wildcard pattern, or regular expression (regex) to use for the selection. Blocks are separated by a colon, wildcards are just an asterisk, and regex patterns start with a tilde:

hostname:groupname:*.example:~(web|db)\.example\.com 

Advanced usage can include group index selections or even ranges within a group:

webservers[0]:webservers[2:4] 

Each block is treated as an inclusion block; that is, all the hosts found in the first pattern are added to all the hosts found in the next pattern, and so on. However, this can be manipulated with control characters to change their behavior. The use of an ampersand defines an inclusion-based selection (all the hosts that exist in both patterns).

The use of an exclamation point defines an exclusion-based selection (all the hosts that exist in the previous patterns but are NOT in the exclusion pattern):

• webservers:&dbservers: Hosts must exist in both the webservers and dbservers groups.
• webservers:!dbservers: Hosts must exist in the webservers group but not the dbservers group.

Once Ansible parses the patterns, it will then apply restrictions if there are any. Restrictions come in the form of limits or failed hosts. This result is stored for the duration of the play, and it is accessible via the play_hosts variable. As each task is executed, this data is consulted, and an additional restriction could be placed upon it to handle serial operations. As failures are encountered, be it a failure to connect or a failure to execute a task, the failed host is placed in a restriction list so that the host will be bypassed in the next task.

If, at any time, a host selection routine gets restricted down to zero hosts, the play execution will stop with an error. A caveat here is that if the play is configured to have a max_fail_precentage or any_errors_fatal parameter, then the playbook execution stops immediately after the task where this condition is met.

Play and task names

While not strictly necessary, it is a good practice to label your plays and tasks with names. These names will show up in the command-line output of ansible-playbook and will show up in the log file if the output of ansible-playbook is directed to log to a file. Task names also come in handy when you want to direct ansible-playbook to start at a specific task and to reference handlers.

There are two main points to consider when naming plays and tasks:

• The names of the plays and tasks should be unique.
• Beware of the kinds of variables that can be used in play and task names.

In general, naming plays and tasks uniquely is a best practice that will help to quickly identify where a problematic task could be residing in your hierarchy of playbooks, roles, task files, handlers, and more. When you first write a small monolithic playbook, they might not seem that important. However, as your use of and confidence in Ansible grows, you will quickly be glad that you named your tasks! Uniqueness is more important when notifying a handler or when starting at a specific task. When task names have duplicates, the behavior of Ansible could be non-deterministic, or at least non-obvious.

With uniqueness as a goal, many playbook authors will look to variables to satisfy this constraint. This strategy might work well, but authors need to be careful regarding the source of the variable data they are referencing. Variable data can come from a variety of locations (which we will cover later in this chapter), and the values assigned to variables can be defined a variety of times. For the sake of play and task names, it is important to remember that only variables for which the values can be determined at the playbook parse time will parse and render correctly. If the data of a referenced variable is discovered via a task or other operation, the variable string will be displayed as unparsed in the output. Let's take a look at an example playbook that utilizes variables for play and task names:

---
- name: play with a {{ var_name }}
  hosts: localhost
  gather_facts: false
  vars:
  - var_name: not-mastery
  tasks:
  - name: set a variable
    ansible.builtin.set_fact:
      task_var_name: "defined variable"
  - name: task with a {{ task_var_name }}
    ansible.builtin.debug:
      msg: "I am mastery task"
- name: second play with a {{ task_var_name }}
  hosts: localhost
  gather_facts: false
  tasks:
  - name: task with a {{ runtime_var_name }}
    ansible.builtin.debug:
      msg: "I am another mastery task" 

At first glance, you might expect at least var_name and task_var_name to render correctly. We can clearly see task_var_name being defined before its use. However, armed with our knowledge that playbooks are parsed in their entirety before execution, we know better. Run the example playbook with the following command:

ansible-playbook -i mastery-hosts -c local names.yaml

The output should look something like Figure 1.10:

Figure 1.10 – A playbook run showing the effect of using variables in task names when they are not defined prior to execution

As you can see in Figure 1.10, the only variable name that is properly rendered is var_name, as it was defined as a static play variable.