Making modules

One of the most important things you can do to make your Puppet manifests clearer and more maintainable is to organize them into modules.

Modules are self-contained bundles of Puppet code that include all the files necessary to implement a thing. Modules may contain flat files, templates, Puppet manifests, custom fact declarations, augeas lenses, and custom Puppet types and providers.

Separating things into modules makes it easier to reuse and share code; it's also the most logical way to organize your manifests. In this example, we'll create a module to manage memcached, a memory caching system commonly used with web applications.

How to do it...

Following are the steps to create an example module:

1. We will use Puppet's module subcommand to create the directory structure for our new module, in our home directory (/home/vagrant):

[t@cookbook ~]$ puppet module generate thomas-memcached

2. We need to create a metadata.json file for this module. Please answer the following questions; if the question is not applicable to this module, feel free to leave it blank:

Puppet uses Semantic Versioning (semver.org) to version modules.
What version is this module? [0.1.0]
-->
Who wrote this module? [thomas]
-->
What license does this module code fall under? [Apache-2.0]
-->
How would you describe this module in a single sentence?
--> A module to install memcached
Where is this module's source code repository?
--> github.com/uphillian/thomas-memcached
Where can others go to learn more about this module? [https://github.com/uphillian/thomas-memcached]
-->
Where can others go to file issues about this module? [https://github.com/uphillian/thomas-memcached/issues]
-->
{
"name": "thomas-memcached",
"version": "0.1.0",
"author": "thomas",
"summary": "A module to install memcached",
"license": "Apache-2.0",
"source": "github.com/uphillian/thomas-memcached",
"project_page": "https://github.com/uphillian/thomas
memcached",
"issues_url": "https://github.com/uphillian/thomas-memcached/issues",
"dependencies": [
{
"name": "puppetlabs-stdlib",
"version_requirement": ">= 1.0.0"
}
]
"data_provider": null
}
----------------------------------------
About to generate this metadata; continue? [n/Y]
--> y
Notice: Generating module at /home/vagrant/memcached...
Notice: Populating templates...
Finished; module generated in memcached.
memcached/spec
memcached/spec/spec_helper.rb
memcached/spec/classes
memcached/spec/classes/init_spec.rb
memcached/metadata.json
memcached/manifests
memcached/manifests/init.pp
memcached/Gemfile
memcached/examples
memcached/examples/init.pp
memcached/README.md
memcached/Rakefile

This command creates the module directory and creates some empty files as starting points.

3. Now, edit memcached/manifests/init.pp and change the class definition at the end of the file to the following. Note that the puppet module created many lines of comments; in a production module, you would want to edit those default comments:

class memcached {
  package { 'memcached': ensure => installed, }
  file { '/etc/memcached.conf':
    source  => 'puppet:///modules/memcached/memcached.conf',
    owner   => 'root',
    group   => 'root',
    mode    => '0644',
    require => Package['memcached'],
 }
 service { 'memcached':
   ensure  => running,
   enable  => true,
   require => [Package['memcached'],
               File['/etc/memcached.conf']],
 }
}

4. Create the modules/thomas-memcached/files directory and then create a file named memcached.conf with the following contents:

[t@cookbook memcached]$ mkdir files
[t@cookbook memcached]$ echo "-m 64 -p 11211 -u nobody -l 127.0.0.1" > files/memcached.conf

5. We would like this module to install memcached. We'll need to run Puppet with root privileges, and we'll use sudo for that. We'll need Puppet to be able to find the module in our home directory; we can specify this on the command line when we run Puppet, as shown in the following code snippet:

t@cookbook:memcached$ sudo /opt/puppetlabs/bin/puppet apply --modulepath=/home/vagrant -e 'include memcached'
Warning: ModuleLoader: module 'memcached' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
  (file & line not available)
Notice: Compiled catalog for cookbook.strangled.net in environment production in 0.46 seconds
Notice: /Stage[main]/Memcached/Package[memcached]/ensure: created
Notice: /Stage[main]/Memcached/File[/etc/memcached.conf]/ensure: defined content as '{md5}febccf4a987759cf4f1558cc625fbea9'
Notice: /Stage[main]/Memcached/Service[memcached]/ensure: ensure changed 'stopped' to 'running'
Notice: Applied catalog in 6.99 seconds

6. We can verify that memcached is running using systemctl or puppet resource:

t@cookbook:memcached$ sudo /opt/puppetlabs/bin/puppet resource service memcached
service { 'memcached':
 ensure => 'running',
 enable => 'true',
}
t@cookbook:memcached$ sudo systemctl status memcached
memcached.service - Memcached
  Loaded: loaded (/usr/lib/systemd/system/memcached.service; enabled; vendor preset: disabled)
  Active: active (running) since Thu 2017-12-28 05:17:41 UTC; 3min 28s ago
Main PID: 4057 (memcached)
  CGroup: /system.slice/memcached.service
          └─4057 /usr/bin/memcached -u memcached -p 11211 -m 64 -c 1024

Dec 28 05:17:41 cookbook systemd[1]: Started Memcached.
Dec 28 05:17:41 cookbook systemd[1]: Starting Memcached...
Note that /opt/puppetlabs/bin/puppet may not be in root's path, use the full path or add the path to a file in /etc/profile.d.  

How it works...

When we created the module using Puppet's module generate command, we used the name thomas-memcached. The name before the hyphen is your username or your username on Puppet forge (an online repository of modules). Modules have a specific directory structure. Not all of these directories need to be present, but if they are, this is how they should be organized:

modules/
└MODULE_NAME/ never use a dash (-) in a module name
└examples/ example usage of the module
└files/ flat files used by the module
└lib/
  └facter/ define new facts for facter
  └puppet/
    └parser/
      └functions/ define a new puppet function, like sort()
    └provider/ define a provider for a new or existing type
    └util/ define helper functions (in ruby)
    └type/ define a new type in puppet
└manifests/
└init.pp class MODULE_NAME { }
└spec/ rSpec tests
└templates/ EPP or ERB template files used by the module

All manifest files (those containing Puppet code) live in the manifests directory. In our example, the memcached class is defined in the manifests/init.pp file, which will be imported automatically.

Inside the memcached class, we refer to the memcached.conf file:

file { '/etc/memcached.conf':
  source => 'puppet:///modules/memcached/memcached.conf',
}

The preceding source parameter tells Puppet to look for the file in:

MODULEPATH/ (/home/vagrant/)
 └memcached/
   └files/
     └memcached.conf

There's more...

Learn to love modules because they'll make your Puppet life a lot easier. They're not complicated, however; practice and experience will help you judge when things should be grouped into modules, and how best to arrange your module structure. Modules can hold more than manifests and files, as we'll see in the next two sections.

Templates

If you need to use a template as a part of the module, place it in the module's templates directory and refer to it as follows:

file { '/etc/memcached.conf':
  content => epp('memcached/memcached.conf.epp),
}

Puppet will look for the file in:

MODULEPATH/memcached/templates/memcached.conf.epp

Facts, functions, types, and providers

Modules can also contain custom facts, custom functions, custom types, and providers. For more information about these, refer to Chapter 9, External Tools and the Puppet Ecosystem.

Third-party modules

You can download modules provided by other people and use them in your own manifests just like the modules you create. For more on this, see Chapter 7, Using Public Modules.

Module organization

For more details on how to organize your modules, see the puppetlabs website: https://puppet.com/docs/puppet/latest/modules_fundamentals.html.

See also

• The Creating custom facts recipe in Chapter 9, External Tools and the Puppet Ecosystem
• The Using public modules recipe in Chapter 9, External Tools and the Puppet Ecosystem
• The Creating your own resource types recipe in Chapter 9, External Tools and the Puppet Ecosystem
• The Creating your own providers recipe in Chapter 9, External Tools and the Puppet Ecosystem