This section will show you how to use the core resource types included within Puppet to reduce the amount of manual work and rework you do every day. By the end of this section, you'll have puppet maintaining some services in your system.
At this point in the process, it would be essential for me to tell you how to ensure that Puppet runs after reboot. It would be chkconfig on the RHEL systems, or update-rc.d on Debian, or smf on Solaris. But hey, why don't we use Puppet itself and let you see how Puppet saves you time and effort?
Use your favorite editor to edit /etc/puppet/manifests/site.pp. Add the following code:
node default {
service { 'puppet':
ensure => running,
enable => true,
}
}Note
/etc/puppet/manifests/site.pp is the one and only filename known by the Puppet server. All other files will be sourced or named from this file. I refer to this file as the origin of the Puppet policy.
ensure => running means that the Puppet agent daemon should be running. This might sound like a catch-22, but it's not really. You can run the Puppet agent manually (we will do this throughout the book.) This service resource ensures that the daemon is running, and starts it if it is not.
enable => true ensures that the process is configured to start up at boot time. This means different things on different platforms, but Puppet allows us to not focus on those details.
Now go to a client node and run the following command. You will see it apply this policy to the system:
$ sudo puppet agent –-test
You will get the following response:
Info: Retrieving plugin Info: Caching catalog for myhost.example.net Info: Applying configuration version '1357518358' Notice: /Stage[main]//Node[default]/Service[puppet]/ensure: ensure changed 'stopped' to 'running' Notice: Finished catalog run in 0.95 seconds
If you want to stop the Puppet daemon, you could easily change the policy to the following code. During the next Puppet run it will disable the service and stop the daemon:
service { 'puppet':
ensure => stopped,
enable => false,
}Note
Let's stop for a moment and talk about the catalog, which is mentioned in the second line of preceding output. The catalog is a distilled, compiled version of the Puppet policy which contains only those directives necessary for the client node.
Obviously our previously mentioned test policy would apply to every node, so they would receive very similar catalogs. We will show you how to differentiate the policy given to each node in step 3.
You can't start services if their packages aren't installed. So why don't we tweak the default node and add a little bit more code? The following segment ensures that the Puppet package is installed, and is the latest version. If the Puppet package is updated, the Puppet service will restart itself to get the new version:
package { 'puppet':
ensure => latest,
notify => Service['puppet'],
}
service { 'puppet':
ensure => running,
enable => true,
subscribe => Package['puppet'],
}Notice how easy it is to set up dependencies between items? notify and subscribe are actually redundant here. You only need one or the other to build the linkage between these two resources. But if you're a belt-and-suspenders kind of person, it never hurts to list the dependency in both the places.
Note
We have introduced another important concept here. By default, declarations in the Puppet policy can happen in any order. You use statements like require, before, notify, and subscribe to define dependencies that ensure that resources are applied in the appropriate order. We will cover ordering and dependencies in the next section.
Now let's go a little further and customize the policy for one node. In the following section, we are going to ensure that the Puppet server is running the latest version of the Puppet master. Just below the default node block, let's add the following code:
node puppet-server-name inherits default {
package { 'puppet-server':
ensure => latest,
notify => Service['puppetmaster'],
}
service { 'puppetmaster':
ensure => running,
enable => true,
subscribe => Package['puppet-server'],
}
}The inherits syntax of the node line says "do everything most nodes do, and also include this policy". You could leave off inherits default and the node would only execute the policy specifically within the node block. In this step we have now customized the catalog, which will be given to this one node.
Let's enable file synchronization down to the client node. First, we need to enable the file server. Use your favorite editor to edit /etc/puppet/fileserver.conf:
[files] path /etc/puppet/files allow *
Now let's create some test files for this example. We'll make the directory owned by you so that we don't have to sudo every command:
$ sudo mkdir /etc/puppet/files $ sudo chown `id –u` /etc/puppet/files $ cp /etc/hosts /etc/puppet/files/ $ cp /etc/motd /etc/puppet/files/
Now let's put some file resources here as examples. You can put the following policy text in the default node block (for all hosts), or you can put it inside a node block for a single host:
file { '/tmp/hosts':
ensure => file,
owner => nobody,
group => nobody,
mode => 0444,
force => false,
source => 'puppet:///files/hosts',
}
file { '/tmp/hosts.linked':
ensure => link,
target => '/tmp/hosts',
}
file { '/tmp/puppet-files':
ensure => directory,
owner => root,
group => root,
mode => 0444,
recurse => true,
source => 'puppet:///files',
}Now let's test out the revised policy on your node:
$ sudo puppet agent --test
You will get the following response:
Info: Retrieving plugin
Info: Caching catalog for myhost.example.com
Info: Applying configuration version '1357520339'
Notice: /Stage[main]//Node[myhost]/File[/tmp/hosts]/ensure: defined content as '{md5}41e6824ef17c17aa577cd6fd6f794351'
Notice: /Stage[main]//Node[myhost]/File[/tmp/hosts.linked]/ensure: created
Notice: /Stage[main]//Node[myhost]/File[/tmp/puppet-files]/ensure: created
Notice: /File[/tmp/puppet-files/hosts]/ensure: defined content as '{md5}41e6824ef17c17aa577cd6fd6f794351'
Notice: /File[/tmp/puppet-files/motd]/ensure: defined content as '{md5}d41d8cd98f00b204e9800998ecf8427e'
Notice: Finished catalog run in 0.51 secondsWith a simple ls –la /tmp you'll be able to see the file, link, and directory that this policy has created.
At this point we've created a small but very effective Puppet policy. You have three different ways to copy files down to hosts. Packages are installed, and services are stopped or started based on your policy. And yet, the entire policy as expressed in the site.pp file, still fits on a single page. In this you can start to perceive the power of Puppet.
Modules are self-contained classes that provide new resources, facts, types, providers, and functions for use in your puppet policy. Each module has its own namespace, so its variables will not conflict with the variables within your policy, or within another module. You can create your own modules, or you can download them from the Puppet Forge.
Module names should only contain lowercase letters, numbers, and underscores, and should begin with a lowercase letter. There is a reason for the last requirement, which I will explain later.
Modules are installed in your module path, usually /etc/puppet/modules.
Now we shall take the examples from our previous section, and build a complete module for maintaining the puppet configuration on your clients and the server. This idea could be easily expanded to maintain any other application. The first step is to build the correct directory structure for a module. The following puppet command will create all of the necessary files and directories for a module to operate as expected:
$ cd /etc/puppet/modules $ puppet module generate my-puppet Notice: Generating module at /etc/puppet/modules/my-puppet my-puppet my-puppet/spec my-puppet/spec/spec_helper.rb my-puppet/README my-puppet/manifests my-puppet/manifests/init.pp my-puppet/tests my-puppet/tests/init.pp my-puppet/Modulefile $ mv my-puppet puppet ##(see tip below)## $ cd puppet $ mkdir files templates $ cd manifests
Note
For reasons unclear to me, the Puppet module will only generate modules with a dash in the name, but dashes are not legal in module names used in your policy. So we rename the module immediately after creating it.
The one file that must exist in every module is manifests/init.pp. This file must contain a class with the same name as the module. Use your favorite editor and open that file now. Set up the initial contents as follows:
class puppet(
$version = 'latest',
$status = 'running',
$environment = 'production',
$server = 'puppet.example.net',
) {
package { 'puppet':
ensure => $version,
notify => Service['puppet']
}
service { 'puppet':
ensure => ${status},
enable => true,
require => Package['puppet'],
}
}You will observe that this is much like the Puppet policy we used in site.pp. You will also note that we have supplied four new variables but only used one of them. We will get to that shortly. For now, let's go back to site.pp and remove this policy. In site.pp, we will replace that with the following invocation of our module:
node default {
class { 'puppet': }
}Much shorter and easier to read, isn't it? We have replaced a block of code with a simple module invocation. This is the main benefit of modules; they allow you to make extensive changes with small and easy-to-read policy statements.
Now, what were those variables we declared before? Those are called class parameters. They define the information that is required for the class to operate. You will notice that in our example, all four parameters were given values. This allows someone to use the class without supplying any information. They will receive the default values that we specify here. If we did not supply a default value, the parameter would be required, and invoking the class without the parameter would create an error.
To supply a parameter in your policy, just add it to the class invocation as follows:
node default {
class { 'puppet':
version => '3.2.0'
}
}Within a module you can create other classes. These are the supporting classes which make up the module. You can use them within the main module class, or you can invoke them directly in the policy. We shall create a member class now, which we shall use for a more specialized configuration. Let's take the remaining Puppet policy out of site.pp, and add it to our server.pp file here:
class puppet::server(
$version = 'latest',
$status = 'running',
$onboot = true,
) {
package { 'puppet-server':
ensure => $version,
notify => Service['puppetmaster'],
}
service { 'puppetmaster':
ensure => $status,
enable => $onboot,
}
}Now, go back to your site.pp policy file and change the block for the node running the Puppet server to the following:
node puppet-server-name inherits default {
class { 'puppet::server':
version => '3.2.0',
}
}We have now created a member class that handles the needs of hosts, which are running the Puppet master service. Every host will be assigned the puppet class. Only our server will be assigned the puppet::server class. This type of opt-in logic is much easier to read and maintain than the if/then/else structures based on the hostname.
If you support different operating systems, you may find that some values need to differ from system to system. For example, you may find that the package name for Puppet in Red Hat-derived Linux is puppet, but on FreeBSD you need puppet3. In this case, you would check fact (covered in the next section) to differentiate between the two operating systems. Here is how to use a selector to handle that situation:
$packagename = $operatingsystem ? {
'redhat' => 'puppet',
'freebsd' => 'puppet3',
default => 'puppet',
}
package { ${packagename}:
ensure => $version,
notify => Service['puppet'],
}Puppet also supports the if/else syntax. However, in all the years I've been working with puppet, I have come to recognize that very few situations demand the if statements. An if statement inside a module hides a policy decision in the guts of a module. For most of the time, you should be using the member classes as documented before, so that the policy is clearly spelled out in the site manifest.
One of the things that a module might contain is files or directories, which should be distributed to the clients. In fact, would this not be a great way to synchronize the Puppet configuration on each host? First, let's copy the puppet.conf file we created during the installation into the module's files directory:
$ cd /etc/puppet/modules/puppet/ $ cp /etc/puppet/puppet.conf files/ puppet.conf
Now, let's add some code to the Puppet class to keep this file up-to-date on all systems:
file { '/etc/puppet/puppet.conf':
ensure => file,
source => 'puppet:///modules/puppet/puppet.conf',
owner => root,
group => root,
mode => 0444,
force => true,
notify => Service['puppet'],
}Now every client will receive exactly the same puppet.conf file. Any changes you make to this file will be distributed to every host that is assigned to the Puppet class.
What if you need to have some differences in the configuration on each host? And what are we going to do with those parameters at the top of the class? Templates are the answer to both of those questions. We will use those parameters to build a custom Puppet configuration based entirely on class parameters.
The first thing we will do is copy the Puppet configuration file to the templates directory:
$ cd /etc/puppet/modules/puppet/ $ cp /etc/puppet/puppet.conf templates/puppet.conf.erb
Next, we shall edit the file to use the variables defined at the top of the class. Open this file in your favorite editor and add or modify the following lines in the [agent] section:
server = <%= @server => environment = <%= @environment =>
Finally, let's modify the code to the Puppet class to build the contents of this file from the template:
file { '/etc/puppet/puppet.conf':
ensure => file,
content => template('puppet/puppet.conf.erb'),
owner => root,
group => root,
mode => 0444,
force => true,
notify => Service['puppet'],
}You have now defined a custom template, which can be edited entirely by statements in the site manifest as follows:
node /testlab/ {
class { 'puppet':
server => 'labhost.example.net',
environment => 'staging',
}
}You have probably noticed from the preceding examples that in most situations the Puppet resources are lowercase, but in some cases the resource types are capitalized. Why is that? When should you use upper or lowercase for the resource types?
When you define new resource, you must use lowercase:
service { 'puppet: ensure => running }When you later refer to that specific resource, you capitalize the resource type. Think of this as a proper noun. In the preceding example you have created a service. After you have created and named this service, you use the capitalized form to refer to it:
package { 'puppet:
ensure => present,
notify => Service['puppet],
}A lowercase resource declares a specific instance. A capitalized resource type refers to an existing resource defined somewhere else. Okay, the proper name analogy isn't quite accurate because we capitalize the resource type, not the resource title. However, it remains the easiest way to think of it.
You will also use uppercase when you want to define some default attributes for a resource. In this situation you use the capitalized resource type without a title. Here are some examples of defaults for some resources:
User {
managehome => true,
shell => '/bin/bash',
}
Package {
schedule => 'daily',
}With these definitions, all the packages in a module will be checked in the daily schedule, unless explicitly defined otherwise, and all users will have the Bash shell unless explicitly overridden. This can save you a great deal of redundant lines in some circumstances.
You may want to limit how often a puppet resource is applied. For example, the Puppet Exec resource type allows you to run an arbitrary command. Unfortunately, the Exec resource will run during every Puppet run. There are two ways to limit how often a resource is applied: run it only when notified, or limit the runs by a schedule. Here are some examples:
You can tell a resource to only apply itself to the system when another resource notifies it. This is done by defining the refreshonly attribute. Here's an example that updates a file containing a list of installed packages every time a new package is installed:
# This defines the default, so all packages will notify the exec
Package {
notify => Exec['save-package-list'],
}
exec { 'save-package-list':
command => 'rpm –qa> /var/tmp/packages-installed.txt',
refreshonly => true,
}Since a resource is only applied once, the command will be run only once no matter how many packages were installed during the Puppet run. If we did not have the refreshonly attribute, the command would be run every time the Puppet agent ran.
Another way to limit how often or when a resource is applied is by using a schedule. A schedule allows you to specify the hours a resource may be applied within, and how often within a given period. For example, I prefer to limit the package upgrades to early in the day when people will be around to deal with any issues that may arise:
# Schedule in the early part of the working day (not peak)
schedule { 'early-day':
range => '9 - 13',
period => daily,
periodmatch => number,
}# default for all packages
Package {
schedule => 'early-day',
}You can read more about defining schedules at http://docs.puppetlabs.com/references/latest/type.html#schedule and more about the schedule and refreshonly metaparameters that you can apply to any resource at http://docs.puppetlabs.com/references/latest/metaparameter.html.
If you step back and look at how your site policy site.pp has changed, you will notice that we have done something very important. We have moved all of the nitty-gritty details (such as the exact name of the Puppet package on each platform) out of the site policy, while leaving all the control and characterization there.
A well-built module hides all of the detailed work of implementation, without specifying the policy. The site manifest defines whether to install a module, what version it should be installing, and whether it should be enabled. This makes it to use the site manifest as a documentation for the site configuration. It becomes easy to expand a configuration to include a whole new class of hosts without any changes the modules.
Likewise, you can add an entirely new operating system to your environment without changing your site policy. In that situation you would edit the modules as necessary, with selectors for each distinct characteristic of the new operating system.
The site manifest expresses what you want. The modules are like butlers and maids; components which implement policy without bothering you with the details. You will find that this approach enables you to do more, faster, and easier than ever before.