Using a decent text editor with the plugins that equip you to write well for Puppet is a really  good step toward better quality. There are quite a few options out there, and it's best to use whatever suits your own unique writing style. Personally, I have used Atom (https://atom.io) most successfully, and recently installed it locally on my workstation. I used Eclipse many years ago (this has also been known previously as Geppetto), which I in fact felt was unwieldy due to a large memory footprint. It's also nice to remain fairly handy with Vim, especially for working on the command line server-side, or if you use a Linux OS on your workstation. There's also TextMate, for an macOS X only editor that has all of Apple's look and feel.

Let's take a look at some of the various options for an Integrated Development Machine (IDE) available to us as Puppet developers.

Vim (http://www.vim.org) is, of course, still a mainstay for text file editing. It has a very long history in the Unix world, and it's a very lightweight command-line text editor. Vim is just about as raw a text editor as you can get. It can be used as a lightning fast and efficient IDE if you have the memory and patience to learn the myriad keyboard commands. My advice is to start out with a few basic commands, and make an effort to pick up a few more each time you use Vim.

You can pimp your Vim and make it better suited for editing Puppet manifests. Let's take a look at that, assuming you've just grabbed a fresh Vim installation, and you have Git installed.

Move to your home directory and clone the given repository with the following commands:

cd ~
git clone https://github.com/ricciocri/vimrc .vim
cd .vim
git pull && git submodule init && git submodule update && git submodule status
cd ~
ln -s .vim/.vimrc 

Cloning the repository into your home directory's .vim directory will configure your Vim settings for you. The repository contains several submodules containing the following:

• Pathogen (https://github.com/tpope/vim-pathogen) is Vim guru Tim Pope's general-purpose add-on that allows you to manage your Vim runtimepath with ease and install Vim plugins and runtime files each in their own private directories, rather than having file collisions.
• Vim-puppet (https://github.com/rodjek/vim-puppet) is the original Vim plugin written by Tim Sharpe, making Vim much more Puppet-friendly.
• snipmate.vim (https://github.com/msanders/snipmate.vim) is a Vim script that implements some of TextMate's snippet features for Vim.
• Syntastic (https://github.com/vim-syntastic/syntastic) is a syntax-checking plugin that runs files through external syntax checkers and displays any resulting errors. This can be done from the command line with the pdk validate command, or automatically as files are saved.
• Tabular (https://github.com/godlygeek/tabular) is used to line up your fat arrows (=>) according to the Puppet Style Guide, so that it will pass running the pdk validate command. (We will cover the pdk validate command in full later.)
• vim-fugitive (https://github.com/tpope/vim-fugitive) provides deep Git integration for Vim.

I can't promise this will be a perfect Vim setup for your own personal Vim style, but it will certainly get you on the right path, and you will have Pathogen installed, so you can further tweak your Vim settings until you have it just how you like it.

 You might also want to fork this repository in GitHub, so you can keep all your settings and share them with your team.

TextMate (http://macromates.com) is an macOS X only editor, and there's a TextMate bundle available (https://github.com/masterzen/puppet-textmate-bundle) for editing Puppet manifests. First, install TextMate and Git (available with the command-line developer tools), and follow these commands to set up the Puppet bundle:

$ mkdir ~/temp
$ cd ~/temp
$ git clone https://github.com/masterzen/puppet-textmate-bundle.git Puppet.tmbundle
$ mv ~/temp/Puppet.tmbundle ~/Library/Application\ Support/TextMate/Bundles/
$ rm -fr ~/temp

Now select a manifest and open it with TextMate. In the TextMate dialog, select Puppet and Install Bundle, and you are all ready to rock.

Here's the IDE that I would recommend based on my own personal style, using my MacBook as the host OS. Atom (https://atom.io) is a fully featured IDE described as, A hackable text editor for the 21st Century and contains all the functionality you'd expect: cross-platform, package (that is, plugin) manager, auto-completion, file browser, multiple panes, find and replace, and so on.

GitHub has developed Atom, and they have built it with the goal of combining the convenience of a fully fledged IDE with the deep configurability of a classic but complex editor such as Vim.

There are literally thousands of open source packages that add new features and functionality to Atom, and here are the ones I recommend specifically for Puppet development:

• language-puppet (adds syntax highlighting and snippets to Puppet files)
• linter-puppet-lint (provides linter support to your Puppet manifests)
• aligner-puppet (aligns the fat arrows according to the Puppet Style Guide)
• erb-snippets (snippets and hotkeys for writing Puppet ERB templates)
• linter-js-yaml (parses your YAML files with JS-YAML)
• tree-view-git-status (displays the Git status of files in the tree view)

If you're a developer in the Windows and .NET world, then look no further than the Puppet language support for Visual Studio Code extension (https://marketplace.visualstudio.com/items?itemName=jpogran.puppet-vscode).

It contains all the features you would expect for Puppet development in the Visual Studio IDE: syntax highlighting, code snippets, file validation, linting according to the Puppet Style Guide, IntelliSense for resources and parameters, importing from the puppet resource command, node graph previewing, and now, Puppet Development Kit (PDK) integration.

This section contains a set of recommendations surrounding good module and class design. Bear in mind that Puppet development is, in principle, just like any other type of software development, and we've learned over many years in software development, and especially at O&O software, that certain modular and class design principles make our development better. I also feel that part of our journey toward infrastructure as code is making our Puppet code just as well-designed, structured, and tested as any other application code.

There's a certain class-naming convention that has developed over time within the Puppet community, and it's really worth taking these into account when structuring your classes:

• init.pp: init.pp contains the class named the same as the module, and is the main entry point for the module.
• params.pp: The params.pp pattern (more on this later in the chapter) is an elegant little hack, taking advantage of Puppet's class inheritance behavior. Any of the other classes in the module inherit from the params class, so have their parameters set appropriately.
• install.pp: The resources related to installing the software should be placed in an install class. The install class must be named <modulename>::install and must be located in the install.pp file.
• config.pp: The resources related to configuring the installed software should be placed in a config class. The config class must be named <modulename>::config and must be located in the config.pp file.
• service.pp: The resources related to managing the service for the software should be placed in a service class. The service class must be named <modulename>::service and must be located in the service.pp file.

For software that is configured in a client/server style, see the following:

• <modulename>::client::install and <modulename>::server::install would be the class names for the install.pp file placed in the client and server directories accordingly
• <modulename>::client::config and <modulename>::server::install would be the class names for the config.pp file placed in the client and server directories accordingly
• <modulename>::client::service and <modulename>::server::service would be the class names for the service.pp files placed in the client and server directories accordingly

init.pp should be the single entry point for the module. In this way, someone reviewing the documentation in particular, as well as the code in init.pp, can have a complete overview of the module's behavior.

If you've used encapsulation effectively and used descriptive class names, you can get a very good sense just by looking at init.pp of how the module actually manages the software.

Ideally, you can use your module with a simple include statement, as follows:

include mymodule

You can also use it with the use of a class declaration, as follows:

class {'mymodule':
  myparam => false,
}

The Apache virtual directory style of configuring a number of defined types would be the third way to use your new module:

mymodule::mydefine {‘define1':
  myotherparam => false,
}

The anti-pattern to this recommendation would be to have a number of classes other than init.pp and your defined types with parameters expecting to be set.

As far as possible, Puppet modules should be made up of classes with a single responsibility. In software engineering, we call this high, functional cohesion. Cohesion in software engineering is the degree to which the elements of a certain module belong together. Try to make each class have a single responsibility, and don't arbitrarily mix together unrelated functionalities in your classes.

As far as possible, these classes should use encapsulation to hide the implementation details from the user; for example, users of your module don't need to be aware of individual resource names. In software engineering, we call this encapsulation. For example, in a config class, we can use several resources, but the user doesn't need to know all about them. Rather, they just simply know that they should use the config class for the configuration of the software to work correctly.

Having classes contain other classes can be very useful, especially in larger modules where you want to improve code readability. You can move chunks of functionality into separate files, and then use the contain keyword to refer to these separated chunks of functionality.

If the vast majority of the people using your module will use the module with a certain parameter set, then of course it makes sense to set that parameter with a default.

Carefully think through how your module is used, and put yourself in the position of a nonexpert user of your own module.

Present the available module parameters in a sensible order, with more often accessed settings before least accessed settings, as opposed to some arbitrary order, such as alphabetical order.

In versions of Puppet proper to the new language features which came out in version 4, we would create class parameters with undefined data types, and then, if we were being very nice, we would use the stdlib validate_<datatype> functions to check appropriate values for those variables:

class vhost (
  $servername,
  $serveraliases,
  $port
)
{ ...

Puppet 4 and 5 have an in-built way of defining the data type that a parameterized class accepts. See the following example:

class vhost (
  String  $servername,
  Array   $serveraliases,
  Integer $port
)
{ ...

Some features to improve quality in your Puppet development, such as puppet-lint, puppet-rspec, and commands such as puppet module create have been around for some time, but previously, you had to discover these tools out there in the wild, install them, and figure out how to use them effectively yourself.

Puppet decided back in August 2017 to bring these things all together on the client side and make them a breeze to use with the new Puppet Development Kit version 1.0. I can certainly recall puppet-rspec always took some time to set up and get working correctly. Now it's all really easy.

Let's take a whistle-stop tour of the module development process using the new PDK 1.0.

• Creating the module framework and metadata: The pdk new module  command runs in the same way as the old puppet module create command, as follows:

$ pdk new module zope –-skip-interview

• Creating the init.pp: There is now a set of creation commands for manifests inside modules, as follows:
  • pdk new class (https://puppet.com/docs/pdk/1.0/pdk_reference.html#pdk-new-class-command)
  • pdk new defined_type (https://puppet.com/docs/pdk/1.0/pdk_reference.html#pdk-new-definedtype-command)
  • pdk new task (https://puppet.com/docs/pdk/1.0/pdk_reference.html#pdk-new-task-command)—see Chapter 6, Workflow, for more details on the new Puppet task functionality.

So, just use the name of the module to create init.pp:

$ pdk new class zope

These commands now negate any need for snippets in your text editor to create the comments, declarations, and other boilerplate code.

• Creating further classes: Create any further classes using the same command. See the following example:

$ pdk new class params

As you are working, you can use the new pdk validate command (https://puppet.com/docs/pdk/1.0/pdk_reference.html#pdk-validate-command) to assist with checking that the module compiles, conforms to the Puppet Style Guide, and has valid metadata:

$ pdk validate

The number one most important thing you can do to bring quality to your modules is to test them! Testing really is one of the most important aspects of software quality assurance in any field of software development. In the agile development community, we've been banging on the table about automated testing for more than 10 years!

Puppet RSpec (http://rspec-puppet.com/tutorial) has been allowing the Puppet community to unit test their modules for quite some time, but it's even easier now with the new PDK 1.0, as everything is set up ready, and you can just add your testing code and run the tests.

From a Puppet perspective, unit testing means checking the output from the compiler. Are the resources contained in the compiled relationship resource catalog, and is their order as expected, given the parameters passed and/or facts present?

When you begin to write tests in Puppet-RSpec, it seems at first like all you are doing is rewriting the Puppet manifests in another Ruby-like language. There is, however, really more to it than that. If there is some reasonable complexity to the module's functionality, for example, testing the dynamic content produced by Puppet templates, support for multiple operating systems, and different actions according to the passed parameters, then these tests actually form a safety net when editing or adding new functionality to your modules, protecting against regressions when refactoring, or upgrading to a new Puppet release.

Let's carry on from the previous two sections and use the development kit to unit test our module. Whenever you generate a class using the pdk new class command, PDK creates a corresponding unit test file. This file, located in your module's /spec/classes folder, already includes a template for writing your unit tests (see http://rspec-puppet.com/tutorial). You can then run the tests using the following command:

$ pdk test unit

Be on the lookout for code smells, especially as your Puppet code base ages! The following link is a research project that describes a bunch of Puppet code smells, which is an XP (extreme programming) term meaning code issues—usually meaning either a poor design or implementation: http://www.tusharma.in/wp-content/uploads/2016/03/ConfigurationSmells_preprint.pdf

Let's quickly run through using the Puppeteer Python-based tool used in the preceding research project:

1. Ensure you have the latest Java SDK installed.
2. Move to your workspace directory ~/workspace, and clone the following Git repository:

$ git clone https://github.com/tushartushar/Puppeteer
$ cd Puppeteer

3. Download the PMD tool (https://github.com/pmd/pmd ) and update the path in the shell script. PMD is an extensible static code analyzer with copy-paste-detector (CPD) built-in.
4. Update the folder path where all the Puppet repositories are placed.
5. Execute the cpdRunner.sh shell script to carry out clone detection using the PMD-CPD tool.
6. Update the REPO_ROOT constant in SmellDetector/Constants.py, which represents the folder path where all the Puppet repositories are placed.
7. Execute Puppeteer.py.
8. Analyze Puppet repository with puppet-lint (optional).
9. Execute puppet-lintRunner.py after setting the repository root.
10. Set the repository root in Puppet-lint_aggregator/PLConstants.py.
11. Execute PuppetLintRules.py, it will generate a consolidated summary of the analysis for all the analyzed projects.

Another issue that can often hit you as your Puppet code base ages is unused code in your codebase. But, there's a tool out there in the wild we can use to keep on top of this issue.

puppet-ghostbuster essentially compares what is actually being used (stored in PuppetDB) to what you think you are using (in your code base directory). This give you the opportunity to slash and burn anything that's really unused. This is great from the point of view of software maintainability. A smaller code base is simply cheaper to maintain!

Let's quickly run through using this Ruby gem.

Make the following settings in your environment variables:

• HIERA_YAML_PATH: The location of the hiera.yaml file. It defaults to /etc/puppetlabs/code/hiera.yaml.
• PUPPETDB_URL: The URL or the PuppetDB. It defaults to http://puppetdb:8080.
• PUPPETDB_CACERT_FILE: Your site's CA certificate.
• PUPPETDB_CERT_FILE: A SSL certificate signed by your site's Puppet CA.
• PUPPETDB_KEY_FILE: The private key for that certificate.

Run the command as follows:

$ find . -type f -exec puppet-lint --only-checks ghostbuster_classes,ghostbuster_defines,ghostbuster_facts,ghostbuster_files,ghostbuster_functions,ghostbuster_hiera_files,ghostbuster_templates,ghostbuster_types {} \+

You can add to and remove from the comma-delimited items to check for unused classes, defined types, facts, files, functions, Hiera files, templates, and types.

It maybe goes without saying that there's no reason to reinvent the wheel when you are authoring your Puppet modules. A few minutes in Puppet Forge (https://forge.puppet.com) can really save you days and days of editing. There are, at the time of writing, more than 5,000 Forge modules, so it makes a great deal of sense to leverage all that hard work done by the Puppet community. Search the Forge first for that bit of software; it's more than likely that something already exists.

In my experience, I have found there is often something that almost does the job. Maybe there's a module (usually an unsupported and unapproved one) that maybe, for example, performs the management for the software you require, but it's only for Ubuntu, and you're using Red Hat. It's usually a better approach to fork that module, whatever shape it's in, and work on that, rather than start from scratch.

The best way for me to describe this best practice is to use an anti-pattern as an example.

I once came across a Puppet developer who would start a module completely from scratch, and then copy and paste lines of code from a Forge module into the new module. From then on, that module exists entirely outside the community! It's not a fork even, so to integrate changes that have been made over time from the community becomes a real pain. You would have to cherry-pick those changes to get the functionality into your own, and you will probably still be left with regression problems. Generally, a best practice is to always at the very least fork the Forge module! This means you get the Git history, which often contains the thoughts that have gone into producing that module.

You see, if you were ever a reader of the great book The Cathedral & the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary (https://www.amazon.com/Cathedral-Bazaar-Musings-Accidental-Revolutionary/dp/0596001088), then you will understand that the Linux-orientated philosophy of software development through a bazaar, collaborative working style trumps spinning off development into a cathedral, independent working style. Well, that's my take on this developer's working style. He was working cathedral-ly, as opposed to bazaar-ly. Effectively, you are making the decision to pit your cathedral team against the multitude of the bazaar, and to my mind, that's simply not wise project management when it comes to giving you a competitive advantage in the internet age.

Sometimes, modules on the Forge get a bit out of date. If the metadata for the module is out of date, you can always produce that again using the PDK new module command (https://puppet.com/docs/pdk/1.0/pdk_generating_modules.html#create-a-module-with-pdk) and commit the new metadata.

Of course, to be a great Puppet community member, it would be an even better practice to make pull requests for the changes you have made and contribute to the work of the community.

Another important recommendation is to simply write great documentation. There's nothing worse, I feel, as a developer, than to have to dig into the code to understand how a module works; it's like having to lift the hood of the car to understand how to drive a vehicle!

Get good at writing English to convey technical ideas! I really think it's a skill that every good developer really needs to master.

Puppet modules use markdown for their documentation formatting. So it makes sense to use either a standalone Markdown editor, or some plugins for your IDE, so that you can create your quality documentation appropriately. Following on from our selection of code IDEs that we considered earlier in the chapter, the corresponding markdown plugins follow.

You can use the vim-instant-markdown plugin (https://github.com/suan/vim-instant-markdown) if you're a vim fan.

You can use the TextMate markdown bundle (https://github.com/textmate/markdown.tmbundle) if you enjoy the Apple look and feel of TextMate.

If, like me, you enjoy using Atom, you can use the Markdown Preview Plus package (https://atom.io/packages/markdown-preview-plus).

If you're a developer in the Windows and .NET world, then look no further than the Markdown editor extension (https://marketplace.visualstudio.com/items?itemName=MadsKristensen.MarkdownEditor).

If you would rather use a standalone Markdown editor, I can recommend personally MacDown for macOS X. My (very) short list of standalone Markdown editors for various operating systems follows.

If you're using Linux, then Remarkable is probably the best standalone editor. It also works on Windows. Some of its features include live preview, exporting to PDF and HTML, GitHub markdown, custom CSS, syntax highlighting, and keyboard shortcuts.

If you would rather use a standalone Markdown editor, I can recommend MacDown for macOS X, which is free (open source). It's heavily inspired by Mou, and is designed with web developers in mind. It has configurable syntax highlighting, live preview, and auto-completion. If you're looking for a lean, fast, configurable standalone Markdown editor, this might be the one for you.

Edit the module's metadata.json file to add module dependencies. See the following example:

"dependencies": [
    { "name":" stankevich/python",
       "version_requirement":">= 1.18.x"
     }
  ]

The name key is the name of the requirement, namely, "pe" or "puppet". The version_requirement key is a semver (http://semver.org) value or range. See the following examples:

• 1.18.0
• 1.18.x
• >= 1.18.x
• >=1.18.x <2.x.x

These would all be valid values for version_requirement.

Check the metadata.json file for validity afterwards using the new PDK command, as follows:

$ pdk validate metadata

The great thing about adding module dependencies is the fact that, when you run the puppet module download command, Puppet will download all the module dependencies accordingly.

This section introduces you to adding compatibility data for the module designed for your version of Puppet or Puppet Enterprise and the operating system you want to work with. To begin with, Edit the module's metadata.json file to add compatibility data.

Express the operating systems your module supports in the module's metadata.json, as shown in the following example:

"operatingsystem_support": [
       { "operatingsystem": "RedHat", },
       { "operatingsystem": "Ubuntu", },
]

The Facter facts operatingsystem and operatingsystemrelease are expected. Here's a more complete example:

"operatingsystem_support": [
       {
           "operatingsystem":"RedHat",
           "operatingsystemrelease":[ "5.0", "6.0" ]
       },
       {
           "operatingsystem": "Ubuntu",
           "operatingsystemrelease": [
               "12.04",
               "10.04"
           ]
       }
   ]

Check the metadata.json file for validity afterwards using the new pdk command:

$ pdk validate metadata

The requirements key in the metadata.json file is a list of external requirements for the module in the following format:

"requirements": [ {“name”: “pe”, “version_requirement”: “5.x”}]

name is the name of the requirement, for example "pe" or "puppet". version_requirement can be a semver (http://semver.org) version range, similar to dependencies.

Again, you can check the metadata.json file for validity afterwards using the new PDK command, as follows:

$ pdk validate metadata

For quite some time when module writing, we've been using the params.pp pattern. One class in the module, by convention called <MODULENAME>::params, sets the variables for any of the other classes:

class zope::params {
  $autoupdate = false,
  $default_service_name = 'ntpd',


  case $facts['os']['family'] {
    'AIX': {
      $service_name = 'xntpd'
    }
    'Debian': {
      $service_name = 'ntp'
    }
    'RedHat': {
      $service_name = $default_service_name
    }
  }
}

So, you can see here that we are using some conditional logic depending on the os::family fact, so that the service_name variable can be set appropriately. We are also exposing the autoupdate variable, and giving it a default value.

This params.pp pattern is an elegant little hack, which takes advantage of Puppet's idiosyncratic class inheritance behavior (using inheritance is generally not recommended in Puppet). Then, any of the other classes in the module inherit from the params class, to have their parameters set appropriately, as shown in the following example:

class zope (
  $autoupdate   = $zope::params::autoupdate,
  $service_name = $zope::params::service_name,
) inherits zope::params {
 ...
}

Since the release of Hiera 5, we are able to simplify our module complexity considerably. By using Hiera-based defaults, we can simplify our module's main classes, and they no longer need to inherit from params.pp. Additionally, you no longer need to explicitly set a default value with the = operator in the parameter declaration.

Let's look at the equivalent configuration to the params.pp pattern using Hiera 5.

First of all, in order to use this new functionality, the data_provider key needs to be set to the heira value in the module's metadata.json file:

...
"data_provider": "hiera",
...

Next, we need to add a hiera.yaml file to the root directory of the module:

---
version: 5
defaults:
  datadir: data
  data_hash: yaml_data
hierarchy:
  - name: "OS family"
    path: "os/%{facts.os.family}.yaml"


  - name: "common"
    path: "common.yaml"

We can then add three files to the /data directory (note that the datadir setting in the hiera.yaml file). The first file of these three is used to set the AIX service_name variable:

# zope/data/os/AIX.yaml
---
zope::service_name: xntpd

The second file is used to set the Debian service_name variable:

# zope/data/os/Debian.yaml
zope::service_name: ntp

And finally, there is the common file, and Hiera will fall through to this file to find its values if it doesn't find a corresponding operating system file when looking for the service_name setting, or a value for autoupdate when searching the previous two files:

# ntp/data/common.yaml
---
ntp::autoupdate: false
ntp::service_name: ntpd

We will look at Hiera 5 in much more detail in Chapter 4, Hiera 5.

In this chapter, we have covered a lot of ground, and I've introduced a bunch of best practices you can use to produce better quality component modules.

In the next chapter, we'll still be covering development in Puppet DSL, and turn our attention to two special modules: role and profile, which can help us to build reusable, configurable, and refactorable site-wide configuration code.