Book Image

DevOps: Puppet, Docker, and Kubernetes

By : Neependra Khare, Ke-Jou Carol Hsu, Hideto Saito, John Arundel, Hui-Chuan Chloe Lee, Thomas Uphill
Book Image

DevOps: Puppet, Docker, and Kubernetes

By: Neependra Khare, Ke-Jou Carol Hsu, Hideto Saito, John Arundel, Hui-Chuan Chloe Lee, Thomas Uphill

Overview of this book

With so many IT management and DevOps tools on the market, both open source and commercial, it’s difficult to know where to start. DevOps is incredibly powerful when implemented correctly, and here’s how to get it done.This Learning Path covers three broad areas: Puppet, Docker, and Kubernetes. This Learning Path is a large resource of recipes to ease your daily DevOps tasks. We begin with recipes that help you develop a complete and expert understanding of Puppet’s latest and most advanced features. Then we provide recipes that help you efficiently work with the Docker environment. Finally, we show you how to better manage containers in different scenarios in production using Kubernetes. This course is based on these books: 1. Puppet Cookbook, Third Edition 2. Docker Cookbook 3. Kubernetes Cookbook
Table of Contents (6 chapters)

Sometimes your Puppet manifest doesn't do exactly what you expected, or perhaps someone else has checked in changes you didn't know about. Either way, it's good to know exactly what Puppet is going to do before it does it.

When you are retrofitting Puppet into an existing infrastructure you might not know whether Puppet is going to update a config file or restart a production service. Any such change could result in unplanned downtime. Also, sometimes manual configuration changes are made on a server that Puppet would overwrite.

To avoid these problems, you can use Puppet's noop mode, which means no operation or do nothing. When run with the noop option, Puppet only reports what it would do but doesn't actually do anything. One caveat here is that even during a noop run, pluginsync still runs and any lib directories in modules will be synced to nodes. This will update external fact definitions and possibly Puppet's types and providers.

You can also use noop mode as a simple auditing tool. It will tell you whether any changes have been made to the machine since Puppet last applied its manifest. Some organizations require all config changes to be made with Puppet, which is one way of implementing a change control process. Unauthorized changes to the resources managed by Puppet can be detected using Puppet in noop mode and you can then decide whether to merge the changes back into the Puppet manifest or undo them.

You can also use the --debug switch when running puppet agent to see the details of every change Puppet makes during an agent run. This can be helpful when trying to figure out how Puppet is applying certain exec resources or to see in what order things are happening.

If you are running a master, you can compile the catalog for a node on the master with the --trace option in addition to --debug. If the catalog is failing to compile, this method will also fail to compile the catalog (if you have an old definition for the cookbook node that is failing, try commenting it out before running this test). This produces a lot of debugging output. For example, to compile the catalog for our cookbook host on our master and place the results into /tmp/cookbook.log:

  • The Auditing resources recipe in Chapter 6, Managing Resources and Files
  • The Automatic syntax checking with Git hooks recipe in Chapter 2, Puppet Infrastructure
  • The Generating reports recipe in this chapter
  • The Testing your Puppet manifests with rspec-puppet recipe in Chapter 9, External Tools and the Puppet Ecosystem
How to do it...

You may run

You can also use noop mode as a simple auditing tool. It will tell you whether any changes have been made to the machine since Puppet last applied its manifest. Some organizations require all config changes to be made with Puppet, which is one way of implementing a change control process. Unauthorized changes to the resources managed by Puppet can be detected using Puppet in noop mode and you can then decide whether to merge the changes back into the Puppet manifest or undo them.

You can also use the --debug switch when running puppet agent to see the details of every change Puppet makes during an agent run. This can be helpful when trying to figure out how Puppet is applying certain exec resources or to see in what order things are happening.

If you are running a master, you can compile the catalog for a node on the master with the --trace option in addition to --debug. If the catalog is failing to compile, this method will also fail to compile the catalog (if you have an old definition for the cookbook node that is failing, try commenting it out before running this test). This produces a lot of debugging output. For example, to compile the catalog for our cookbook host on our master and place the results into /tmp/cookbook.log:

  • The Auditing resources recipe in Chapter 6, Managing Resources and Files
  • The Automatic syntax checking with Git hooks recipe in Chapter 2, Puppet Infrastructure
  • The Generating reports recipe in this chapter
  • The Testing your Puppet manifests with rspec-puppet recipe in Chapter 9, External Tools and the Puppet Ecosystem
How it works...

In the noop mode, Puppet

You can also use noop mode as a simple auditing tool. It will tell you whether any changes have been made to the machine since Puppet last applied its manifest. Some organizations require all config changes to be made with Puppet, which is one way of implementing a change control process. Unauthorized changes to the resources managed by Puppet can be detected using Puppet in noop mode and you can then decide whether to merge the changes back into the Puppet manifest or undo them.

You can also use the --debug switch when running puppet agent to see the details of every change Puppet makes during an agent run. This can be helpful when trying to figure out how Puppet is applying certain exec resources or to see in what order things are happening.

If you are running a master, you can compile the catalog for a node on the master with the --trace option in addition to --debug. If the catalog is failing to compile, this method will also fail to compile the catalog (if you have an old definition for the cookbook node that is failing, try commenting it out before running this test). This produces a lot of debugging output. For example, to compile the catalog for our cookbook host on our master and place the results into /tmp/cookbook.log:

  • The Auditing resources recipe in Chapter 6, Managing Resources and Files
  • The Automatic syntax checking with Git hooks recipe in Chapter 2, Puppet Infrastructure
  • The Generating reports recipe in this chapter
  • The Testing your Puppet manifests with rspec-puppet recipe in Chapter 9, External Tools and the Puppet Ecosystem
There's more...

You can also use noop mode as a

simple auditing tool. It will tell you whether any changes have been made to the machine since Puppet last applied its manifest. Some organizations require all config changes to be made with Puppet, which is one way of implementing a change control process. Unauthorized changes to the resources managed by Puppet can be detected using Puppet in noop mode and you can then decide whether to merge the changes back into the Puppet manifest or undo them.

You can also use the --debug switch when running puppet agent to see the details of every change Puppet makes during an agent run. This can be helpful when trying to figure out how Puppet is applying certain exec resources or to see in what order things are happening.

If you are running a master, you can compile the catalog for a node on the master with the --trace option in addition to --debug. If the catalog is failing to compile, this method will also fail to compile the catalog (if you have an old definition for the cookbook node that is failing, try commenting it out before running this test). This produces a lot of debugging output. For example, to compile the catalog for our cookbook host on our master and place the results into /tmp/cookbook.log:

  • The Auditing resources recipe in Chapter 6, Managing Resources and Files
  • The Automatic syntax checking with Git hooks recipe in Chapter 2, Puppet Infrastructure
  • The Generating reports recipe in this chapter
  • The Testing your Puppet manifests with rspec-puppet recipe in Chapter 9, External Tools and the Puppet Ecosystem
See also

The Auditing resources recipe in
  • Chapter 6, Managing Resources and Files
  • The Automatic syntax checking with Git hooks recipe in Chapter 2, Puppet Infrastructure
  • The Generating reports recipe in this chapter
  • The Testing your Puppet manifests with rspec-puppet recipe in Chapter 9, External Tools and the Puppet Ecosystem

When you use the exec resources to run commands on the node, Puppet will give you an error message such as the following if a command returns a non-zero exit status:

As you can see, Puppet not only reports that the command failed, but shows its output:

This is useful to figure out why the command didn't work, but sometimes the command actually succeeds (in that it returns a zero exit status) but still doesn't do what we wanted. In that case, how can you see the command output? You can use the logoutput attribute.

How to do it...

Follow these steps in order to log command output:

Define an exec resource with the logoutput parameter as shown in the following code snippet:
exec { 'exec with output':
  command   => '/bin/cat /etc/hostname',
logoutput => true,
}
Run Puppet:
t@mylaptop ~/puppet/manifests $ puppet apply exec.pp
Notice: Compiled catalog for mylaptop in environment production in 0.46 seconds
Notice: /Stage[main]/Main/Exec[exec with outout]/returns: mylaptop
Notice: /Stage[main]/Main/Exec[exec with outout]/returns: executed successfully
Notice: Finished catalog run in 0.06 seconds
As you can see, even though the command succeeds, Puppet prints the output:
mylaptop
How it works...

The logoutput attribute has three possible settings:

false: This never There's more...

You can set the default value of logoutput to always display command output for all exec resources by defining the following in your site.pp file:

Exec { logoutput => true,

It can be very helpful when debugging problems if you can print out information at a certain point in the manifest. This is a good way to tell, for example, if a variable isn't defined or has an unexpected value. Sometimes it's useful just to know that a particular piece of code has been run. Puppet's notify resource lets you print out such messages.

In addition to simple messages, we can output variables within our notify statements. Additionally, we can treat the notify calls the same as other resources, having them require or be required by other resources.

Puppet compiles your manifests into a catalog; the order in which resources are executed on the client (node) may not be the same as the order of the resources within your source files. When you are using a notify resource for debugging, you should use resource chaining to ensure that the notify resource is executed before or after your failing resource.

For example, if the exec failing exec is failing, you can chain a notify resource to run directly before the failed exec resource as shown here:

If you don't chain the resource or use a metaparameter such as before or require, there is no guarantee your notify statement will be executed near the other resources you are interested in debugging. More information on resource ordering can be found at https://docs.puppetlabs.com/puppet/latest/reference/lang_relationships.html.

For example, to have your notify resource run after 'failing exec' in the preceding code snippet, use:

notify { 'Resource X has been applied':
  require => Exec['failing exec'],
}

Note, however, that in this case the notify resource will fail to execute since the exec failed. When a resource fails, all the resources that depended on that resource are skipped:

notify {'failed exec failed': 
  require => Exec['failing exec']
}

When we run Puppet, we see that the notify resource is skipped:

t@mylaptop ~/puppet/manifests $ puppet apply fail.pp
...
Error: /bin/grepmylaptop /etc/hosts returned 1 instead of one of [0]
Error: /Stage[main]/Main/Exec[failing exec]/returns: change from notrun to 0 failed: /bin/grepmylaptop /etc/hosts returned 1 instead of one of [0]
Notice: /Stage[main]/Main/Notify[failed exec failed]: Dependency Exec[failing exec] has failures: true
Warning: /Stage[main]/Main/Notify[failed exec failed]: Skipping because of failed dependencies
Notice: Finished catalog run in 0.06 seconds
How to do it...

Define a notify resource in your manifest at the point you want to investigate:

notify { 'Got this far!': }

In addition to simple messages, we can output variables within our notify statements. Additionally, we can treat the notify calls the same as other resources, having them require or be required by other resources.

Puppet compiles your manifests into a catalog; the order in which resources are executed on the client (node) may not be the same as the order of the resources within your source files. When you are using a notify resource for debugging, you should use resource chaining to ensure that the notify resource is executed before or after your failing resource.

For example, if the exec failing exec is failing, you can chain a notify resource to run directly before the failed exec resource as shown here:

If you don't chain the resource or use a metaparameter such as before or require, there is no guarantee your notify statement will be executed near the other resources you are interested in debugging. More information on resource ordering can be found at https://docs.puppetlabs.com/puppet/latest/reference/lang_relationships.html.

For example, to have your notify resource run after 'failing exec' in the preceding code snippet, use:

notify { 'Resource X has been applied':
  require => Exec['failing exec'],
}

Note, however, that in this case the notify resource will fail to execute since the exec failed. When a resource fails, all the resources that depended on that resource are skipped:

notify {'failed exec failed': 
  require => Exec['failing exec']
}

When we run Puppet, we see that the notify resource is skipped:

t@mylaptop ~/puppet/manifests $ puppet apply fail.pp
...
Error: /bin/grepmylaptop /etc/hosts returned 1 instead of one of [0]
Error: /Stage[main]/Main/Exec[failing exec]/returns: change from notrun to 0 failed: /bin/grepmylaptop /etc/hosts returned 1 instead of one of [0]
Notice: /Stage[main]/Main/Notify[failed exec failed]: Dependency Exec[failing exec] has failures: true
Warning: /Stage[main]/Main/Notify[failed exec failed]: Skipping because of failed dependencies
Notice: Finished catalog run in 0.06 seconds
How it works...

When this resource is

In addition to simple messages, we can output variables within our notify statements. Additionally, we can treat the notify calls the same as other resources, having them require or be required by other resources.

Puppet compiles your manifests into a catalog; the order in which resources are executed on the client (node) may not be the same as the order of the resources within your source files. When you are using a notify resource for debugging, you should use resource chaining to ensure that the notify resource is executed before or after your failing resource.

For example, if the exec failing exec is failing, you can chain a notify resource to run directly before the failed exec resource as shown here:

If you don't chain the resource or use a metaparameter such as before or require, there is no guarantee your notify statement will be executed near the other resources you are interested in debugging. More information on resource ordering can be found at https://docs.puppetlabs.com/puppet/latest/reference/lang_relationships.html.

For example, to have your notify resource run after 'failing exec' in the preceding code snippet, use:

notify { 'Resource X has been applied':
  require => Exec['failing exec'],
}

Note, however, that in this case the notify resource will fail to execute since the exec failed. When a resource fails, all the resources that depended on that resource are skipped:

notify {'failed exec failed': 
  require => Exec['failing exec']
}

When we run Puppet, we see that the notify resource is skipped:

t@mylaptop ~/puppet/manifests $ puppet apply fail.pp
...
Error: /bin/grepmylaptop /etc/hosts returned 1 instead of one of [0]
Error: /Stage[main]/Main/Exec[failing exec]/returns: change from notrun to 0 failed: /bin/grepmylaptop /etc/hosts returned 1 instead of one of [0]
Notice: /Stage[main]/Main/Notify[failed exec failed]: Dependency Exec[failing exec] has failures: true
Warning: /Stage[main]/Main/Notify[failed exec failed]: Skipping because of failed dependencies
Notice: Finished catalog run in 0.06 seconds
There's more...

In addition to simple messages, we can output variables within our notify statements. Additionally, we can treat the notify calls the same as other resources, having them require or be required by other resources.

Puppet compiles your manifests into a catalog; the order in which resources are executed on the client (node) may not be the same as the order of the resources within your source files. When you are using a notify resource for debugging, you should use resource chaining to ensure that the notify resource is executed before or after your failing resource.

For example, if the exec failing exec is failing, you can chain a notify resource to run directly before the failed exec resource as shown here:

If you don't chain the resource or use a metaparameter such as before or require, there is no guarantee your notify statement will be executed near the other resources you are interested in debugging. More information on resource ordering can be found at https://docs.puppetlabs.com/puppet/latest/reference/lang_relationships.html.

For example, to have your notify resource run after 'failing exec' in the preceding code snippet, use:

notify { 'Resource X has been applied':
  require => Exec['failing exec'],
}

Note, however, that in this case the notify resource will fail to execute since the exec failed. When a resource fails, all the resources that depended on that resource are skipped:

notify {'failed exec failed': 
  require => Exec['failing exec']
}

When we run Puppet, we see that the notify resource is skipped:

t@mylaptop ~/puppet/manifests $ puppet apply fail.pp
...
Error: /bin/grepmylaptop /etc/hosts returned 1 instead of one of [0]
Error: /Stage[main]/Main/Exec[failing exec]/returns: change from notrun to 0 failed: /bin/grepmylaptop /etc/hosts returned 1 instead of one of [0]
Notice: /Stage[main]/Main/Notify[failed exec failed]: Dependency Exec[failing exec] has failures: true
Warning: /Stage[main]/Main/Notify[failed exec failed]: Skipping because of failed dependencies
Notice: Finished catalog run in 0.06 seconds
Printing out variable values

You can refer to

Puppet compiles your manifests into a catalog; the order in which resources are executed on the client (node) may not be the same as the order of the resources within your source files. When you are using a notify resource for debugging, you should use resource chaining to ensure that the notify resource is executed before or after your failing resource.

For example, if the exec failing exec is failing, you can chain a notify resource to run directly before the failed exec resource as shown here:

If you don't chain the resource or use a metaparameter such as before or require, there is no guarantee your notify statement will be executed near the other resources you are interested in debugging. More information on resource ordering can be found at https://docs.puppetlabs.com/puppet/latest/reference/lang_relationships.html.

For example, to have your notify resource run after 'failing exec' in the preceding code snippet, use:

notify { 'Resource X has been applied':
  require => Exec['failing exec'],
}

Note, however, that in this case the notify resource will fail to execute since the exec failed. When a resource fails, all the resources that depended on that resource are skipped:

notify {'failed exec failed': 
  require => Exec['failing exec']
}

When we run Puppet, we see that the notify resource is skipped:

t@mylaptop ~/puppet/manifests $ puppet apply fail.pp
...
Error: /bin/grepmylaptop /etc/hosts returned 1 instead of one of [0]
Error: /Stage[main]/Main/Exec[failing exec]/returns: change from notrun to 0 failed: /bin/grepmylaptop /etc/hosts returned 1 instead of one of [0]
Notice: /Stage[main]/Main/Notify[failed exec failed]: Dependency Exec[failing exec] has failures: true
Warning: /Stage[main]/Main/Notify[failed exec failed]: Skipping because of failed dependencies
Notice: Finished catalog run in 0.06 seconds
Resource ordering

Puppet compiles your manifests into

a catalog; the order in which resources are executed on the client (node) may not be the same as the order of the resources within your source files. When you are using a notify resource for debugging, you should use resource chaining to ensure that the notify resource is executed before or after your failing resource.

For example, if the exec failing exec is failing, you can chain a notify resource to run directly before the failed exec resource as shown here:

If you don't chain the resource or use a metaparameter such as before or require, there is no guarantee your notify statement will be executed near the other resources you are interested in debugging. More information on resource ordering can be found at https://docs.puppetlabs.com/puppet/latest/reference/lang_relationships.html.

For example, to have your notify resource run after 'failing exec' in the preceding code snippet, use:

notify { 'Resource X has been applied':
  require => Exec['failing exec'],
}

Note, however, that in this case the notify resource will fail to execute since the exec failed. When a resource fails, all the resources that depended on that resource are skipped:

notify {'failed exec failed': 
  require => Exec['failing exec']
}

When we run Puppet, we see that the notify resource is skipped:

t@mylaptop ~/puppet/manifests $ puppet apply fail.pp
...
Error: /bin/grepmylaptop /etc/hosts returned 1 instead of one of [0]
Error: /Stage[main]/Main/Exec[failing exec]/returns: change from notrun to 0 failed: /bin/grepmylaptop /etc/hosts returned 1 instead of one of [0]
Notice: /Stage[main]/Main/Notify[failed exec failed]: Dependency Exec[failing exec] has failures: true
Warning: /Stage[main]/Main/Notify[failed exec failed]: Skipping because of failed dependencies
Notice: Finished catalog run in 0.06 seconds

If you're managing a lot of machines, Puppet's reporting facility can give you some valuable information on what's actually happening out there.

How to do it...

To enable reports, just add this to a client's puppet.conf: within the [main] or [agent] sections:

report = true
How it works...

With reporting enabled, Puppet will generate a report file, containing data such as:

Date and time of the run
Total time for the run
Log messages output during the run
List of all the resources in the client's manifest
Whether Puppet changed any resources, and how many
Whether the run succeeded or failed

By default, these reports
There's more...

If you have more than one master server, you can have all your reports sent to the same server by specifying report_server in the [agent] section of puppet.conf.

If you just want one report, or you don't want to enable reporting all the time, you can add the --report switch to the command line when you run Puppet agent manually:

[root@cookbook ~]# puppet agent -t --report Notice: Finished catalog run in 0.34 seconds

You won't see any additional output, but a report file will be generated in the report directory.

You can also see some overall statistics about a Puppet run by supplying the --summarize switch:

[root@cookbook ~]# puppet agent -t --report --summarize Notice: Finished catalog run in 0.35 seconds Changes: Total: 2 Events: Total: 2 Success: 2 Resources: Total: 10 Changed: 2 Out of sync: 2 Time: Filebucket: 0.00 Schedule: 0.00 Notify: 0.00 Config retrieval: 0.94 Total: 0.95 Last run: 1416727357 Version: Config: 1416727291 Puppet: 3.7.3

Puppet can generate different types of reports with the reports option in the [main] or [master] section of puppet.conf on your Puppet master servers. There are several built-in report types listed at https://docs.puppetlabs.com/references/latest/report.html. In addition to the built-in report types, there are some community developed reports that are quite useful. The Foreman (http://theforeman.org), for example, provides a Foreman report type that you can enable to forward your node reports to the Foreman.

Other report types

Puppet can generate different types of reports with the reports option in the [main] or [master] section of puppet.conf on your Puppet master servers. There are several built-in report types

listed at https://docs.puppetlabs.com/references/latest/report.html. In addition to the built-in report types, there are some community developed reports that are quite useful. The Foreman (http://theforeman.org), for example, provides a Foreman report type that you can enable to forward your node reports to the Foreman.

See also

The Auditing resources recipe in

As your manifests get bigger and more complex, it can be helpful to create HTML documentation for your nodes and classes using Puppet's automatic documentation tool, puppet doc.

How to do it...

Follow these steps to generate
How it works...

The puppet doc command
There's more...

The puppet doc command will

Dependencies can get complicated quickly, and it's easy to end up with a circular dependency (where A depends on B, which depends on A) that will cause Puppet to complain and stop working. Fortunately, Puppet's --graph option makes it easy to generate a diagram of your resources and the dependencies between them, which can be a big help in fixing such problems.

Follow these steps to generate a dependency graph for your manifest:

  1. Create the directories for a new trifecta module:
    ubuntu@cookbook:~/puppet$ mkdir modules/trifecta
    ubuntu@cookbook:~/puppet$ mkdir modules/trifecta/manifests
    ubuntu@cookbook:~/puppet$ mkdir modules/trifecta/files
    
  2. Create the file modules/trifecta/manifests/init.pp with the following code containing a deliberate circular dependency (can you spot it?):
    class trifecta {
      package { 'ntp':
        ensure  => installed,
        require => File['/etc/ntp.conf'],
      }
    
      service { 'ntp':
        ensure  => running,
        require => Package['ntp'],
      }
    
      file { '/etc/ntp.conf':
        source  => 'puppet:///modules/trifecta/ntp.conf',
        notify  => Service['ntp'],
        require => Package['ntp'],
      }
    }
  3. Create a simple ntp.conf file:
    t@mylaptop~/puppet $ cd modules/trifecta/files
    t@mylaptop~/puppet/modules/trifecta/files $ echo "server 127.0.0.1" >ntp.conf
    
  4. Since we'll be working locally on this problem, create a trifecta.pp manifest that includes the broken trifecta class:
    include trifecta
  5. Run Puppet:
    t@mylaptop ~/puppet/manifests $ puppet apply trifecta.pp
    Notice: Compiled catalog for mylaptop in environment production in 1.32 seconds
    Error: Could not apply complete catalog: Found 1 dependency cycle:
    (File[/etc/ntp.conf] => Package[ntp] => File[/etc/ntp.conf])
    Try the '--graph' option and opening the resulting '.dot' file in OmniGraffle or GraphViz
    
  6. Run Puppet with the --graph option as suggested:
    t@mylaptop ~/puppet/manifests $ puppet apply trifecta.pp --graph
    Notice: Compiled catalog for mylaptop in environment production in 1.26 seconds
    Error: Could not apply complete catalog: Found 1 dependency cycle:
    (File[/etc/ntp.conf] => Package[ntp] => File[/etc/ntp.conf])
    Cycle graph written to /home/tuphill/.puppet/var/state/graphs/cycles.dot.
    Notice: Finished catalog run in 0.03 seconds
    
  7. Check whether the graph files have been created:
    t@mylaptop ~/puppet/manifests $ cd ~/.puppet/var/state/graphs
    t@mylaptop ~/.puppet/var/state/graphs $ ls -l
    total 16
    -rw-rw-r--. 1 thomasthomas  121 Nov 23 23:11 cycles.dot
    -rw-rw-r--. 1 thomasthomas 2885 Nov 23 23:11 expanded_relationships.dot
    -rw-rw-r--. 1 thomasthomas 1557 Nov 23 23:11 relationships.dot
    -rw-rw-r--. 1 thomasthomas 1680 Nov 23 23:11 resources.dot
    
  8. Create a graphic using the dot command as follows:
    ubuntu@cookbook:~/puppet$ dot -Tpng -o relationships.png /var/lib/puppet/state/graphs/relationships.dot
    
  9. The graphic will look something like the this:
    How to do it...

When you run puppet agent --graph (or enable the graph option in puppet.conf), Puppet will generate three graphs in the DOT format (a graphics language):

The dot tool (part of the graphviz package) will convert these to an image format such as PNG for viewing.

In the relationships graph, each resource in your manifest is shown as a balloon (known as a vertex), with arrowed lines connecting them to indicate the dependencies. You can see that in our example, the dependencies between File['/etc/ntp.conf'] and Package['ntp'] are bidirectional. When Puppet tries to decide where to begin applying these resources, it can start at File['/etc/ntp.conf'] and look for what depends on File['/etc/ntp.conf'] and end up at Package['ntp']. When Puppet looks for the dependencies

of Package['ntp'], it will end up back at File['/etc/ntp.conf'], forming a circular path. This type of problem is known as a circular dependency problem; Puppet can't decide where to start because the two resources depend on each other.

To fix the circular dependency problem, all you need to do is remove one of the dependency lines and break the circle. The following code fixes the problem:

Now when we run puppet apply or agent with the --graph option, the resulting graph does not have any circular paths (cycles):

How it works...

In this graph it is easy to see that Package[ntp] is the first resource to be applied, then File[/etc/ntp.conf], and finally Service[ntp].

Getting ready

Install the graphviz package to view the diagram files:

t@mylaptop ~ $ sudo puppet resource package graphviz ensure=installed Notice: /Package[graphviz]/ensure: created package { 'graphviz': ensure => '2.34.0-9.fc20', }

Follow these steps to generate a dependency graph for your manifest:

  1. Create the directories for a new trifecta module:
    ubuntu@cookbook:~/puppet$ mkdir modules/trifecta
    ubuntu@cookbook:~/puppet$ mkdir modules/trifecta/manifests
    ubuntu@cookbook:~/puppet$ mkdir modules/trifecta/files
    
  2. Create the file modules/trifecta/manifests/init.pp with the following code containing a deliberate circular dependency (can you spot it?):
    class trifecta {
      package { 'ntp':
        ensure  => installed,
        require => File['/etc/ntp.conf'],
      }
    
      service { 'ntp':
        ensure  => running,
        require => Package['ntp'],
      }
    
      file { '/etc/ntp.conf':
        source  => 'puppet:///modules/trifecta/ntp.conf',
        notify  => Service['ntp'],
        require => Package['ntp'],
      }
    }
  3. Create a simple ntp.conf file:
    t@mylaptop~/puppet $ cd modules/trifecta/files
    t@mylaptop~/puppet/modules/trifecta/files $ echo "server 127.0.0.1" >ntp.conf
    
  4. Since we'll be working locally on this problem, create a trifecta.pp manifest that includes the broken trifecta class:
    include trifecta
  5. Run Puppet:
    t@mylaptop ~/puppet/manifests $ puppet apply trifecta.pp
    Notice: Compiled catalog for mylaptop in environment production in 1.32 seconds
    Error: Could not apply complete catalog: Found 1 dependency cycle:
    (File[/etc/ntp.conf] => Package[ntp] => File[/etc/ntp.conf])
    Try the '--graph' option and opening the resulting '.dot' file in OmniGraffle or GraphViz
    
  6. Run Puppet with the --graph option as suggested:
    t@mylaptop ~/puppet/manifests $ puppet apply trifecta.pp --graph
    Notice: Compiled catalog for mylaptop in environment production in 1.26 seconds
    Error: Could not apply complete catalog: Found 1 dependency cycle:
    (File[/etc/ntp.conf] => Package[ntp] => File[/etc/ntp.conf])
    Cycle graph written to /home/tuphill/.puppet/var/state/graphs/cycles.dot.
    Notice: Finished catalog run in 0.03 seconds
    
  7. Check whether the graph files have been created:
    t@mylaptop ~/puppet/manifests $ cd ~/.puppet/var/state/graphs
    t@mylaptop ~/.puppet/var/state/graphs $ ls -l
    total 16
    -rw-rw-r--. 1 thomasthomas  121 Nov 23 23:11 cycles.dot
    -rw-rw-r--. 1 thomasthomas 2885 Nov 23 23:11 expanded_relationships.dot
    -rw-rw-r--. 1 thomasthomas 1557 Nov 23 23:11 relationships.dot
    -rw-rw-r--. 1 thomasthomas 1680 Nov 23 23:11 resources.dot
    
  8. Create a graphic using the dot command as follows:
    ubuntu@cookbook:~/puppet$ dot -Tpng -o relationships.png /var/lib/puppet/state/graphs/relationships.dot
    
  9. The graphic will look something like the this:
    How to do it...

When you run puppet agent --graph (or enable the graph option in puppet.conf), Puppet will generate three graphs in the DOT format (a graphics language):

The dot tool (part of the graphviz package) will convert these to an image format such as PNG for viewing.

In the relationships graph, each resource in your manifest is shown as a balloon (known as a vertex), with arrowed lines connecting them to indicate the dependencies. You can see that in our example, the dependencies between File['/etc/ntp.conf'] and Package['ntp'] are bidirectional. When Puppet tries to decide where to begin applying these resources, it can start at File['/etc/ntp.conf'] and look for what depends on File['/etc/ntp.conf'] and end up at Package['ntp']. When Puppet looks for the dependencies

of Package['ntp'], it will end up back at File['/etc/ntp.conf'], forming a circular path. This type of problem is known as a circular dependency problem; Puppet can't decide where to start because the two resources depend on each other.

To fix the circular dependency problem, all you need to do is remove one of the dependency lines and break the circle. The following code fixes the problem:

Now when we run puppet apply or agent with the --graph option, the resulting graph does not have any circular paths (cycles):

How it works...

In this graph it is easy to see that Package[ntp] is the first resource to be applied, then File[/etc/ntp.conf], and finally Service[ntp].

How to do it...

Follow these steps to

generate a dependency graph for your manifest:

  1. Create the directories for a new trifecta module:
    ubuntu@cookbook:~/puppet$ mkdir modules/trifecta
    ubuntu@cookbook:~/puppet$ mkdir modules/trifecta/manifests
    ubuntu@cookbook:~/puppet$ mkdir modules/trifecta/files
    
  2. Create the file modules/trifecta/manifests/init.pp with the following code containing a deliberate circular dependency (can you spot it?):
    class trifecta {
      package { 'ntp':
        ensure  => installed,
        require => File['/etc/ntp.conf'],
      }
    
      service { 'ntp':
        ensure  => running,
        require => Package['ntp'],
      }
    
      file { '/etc/ntp.conf':
        source  => 'puppet:///modules/trifecta/ntp.conf',
        notify  => Service['ntp'],
        require => Package['ntp'],
      }
    }
  3. Create a simple ntp.conf file:
    t@mylaptop~/puppet $ cd modules/trifecta/files
    t@mylaptop~/puppet/modules/trifecta/files $ echo "server 127.0.0.1" >ntp.conf
    
  4. Since we'll be working locally on this problem, create a trifecta.pp manifest that includes the broken trifecta class:
    include trifecta
  5. Run Puppet:
    t@mylaptop ~/puppet/manifests $ puppet apply trifecta.pp
    Notice: Compiled catalog for mylaptop in environment production in 1.32 seconds
    Error: Could not apply complete catalog: Found 1 dependency cycle:
    (File[/etc/ntp.conf] => Package[ntp] => File[/etc/ntp.conf])
    Try the '--graph' option and opening the resulting '.dot' file in OmniGraffle or GraphViz
    
  6. Run Puppet with the --graph option as suggested:
    t@mylaptop ~/puppet/manifests $ puppet apply trifecta.pp --graph
    Notice: Compiled catalog for mylaptop in environment production in 1.26 seconds
    Error: Could not apply complete catalog: Found 1 dependency cycle:
    (File[/etc/ntp.conf] => Package[ntp] => File[/etc/ntp.conf])
    Cycle graph written to /home/tuphill/.puppet/var/state/graphs/cycles.dot.
    Notice: Finished catalog run in 0.03 seconds
    
  7. Check whether the graph files have been created:
    t@mylaptop ~/puppet/manifests $ cd ~/.puppet/var/state/graphs
    t@mylaptop ~/.puppet/var/state/graphs $ ls -l
    total 16
    -rw-rw-r--. 1 thomasthomas  121 Nov 23 23:11 cycles.dot
    -rw-rw-r--. 1 thomasthomas 2885 Nov 23 23:11 expanded_relationships.dot
    -rw-rw-r--. 1 thomasthomas 1557 Nov 23 23:11 relationships.dot
    -rw-rw-r--. 1 thomasthomas 1680 Nov 23 23:11 resources.dot
    
  8. Create a graphic using the dot command as follows:
    ubuntu@cookbook:~/puppet$ dot -Tpng -o relationships.png /var/lib/puppet/state/graphs/relationships.dot
    
  9. The graphic will look something like the this:
    How to do it...

When you run puppet agent --graph (or enable the graph option in puppet.conf), Puppet will generate three graphs in the DOT format (a graphics language):

The dot tool (part of the graphviz package) will convert these to an image format such as PNG for viewing.

In the relationships graph, each resource in your manifest is shown as a balloon (known as a vertex), with arrowed lines connecting them to indicate the dependencies. You can see that in our example, the dependencies between File['/etc/ntp.conf'] and Package['ntp'] are bidirectional. When Puppet tries to decide where to begin applying these resources, it can start at File['/etc/ntp.conf'] and look for what depends on File['/etc/ntp.conf'] and end up at Package['ntp']. When Puppet looks for the dependencies

of Package['ntp'], it will end up back at File['/etc/ntp.conf'], forming a circular path. This type of problem is known as a circular dependency problem; Puppet can't decide where to start because the two resources depend on each other.

To fix the circular dependency problem, all you need to do is remove one of the dependency lines and break the circle. The following code fixes the problem:

Now when we run puppet apply or agent with the --graph option, the resulting graph does not have any circular paths (cycles):

How it works...

In this graph it is easy to see that Package[ntp] is the first resource to be applied, then File[/etc/ntp.conf], and finally Service[ntp].

How it works...

When you

run puppet agent --graph (or enable the graph option in puppet.conf), Puppet will generate three graphs in the DOT format (a graphics language):

The dot tool (part of the graphviz package) will convert these to an image format such as PNG for viewing.

In the relationships graph, each resource in your manifest is shown as a balloon (known as a vertex), with arrowed lines connecting them to indicate the dependencies. You can see that in our example, the dependencies between File['/etc/ntp.conf'] and Package['ntp'] are bidirectional. When Puppet tries to decide where to begin applying these resources, it can start at File['/etc/ntp.conf'] and look for what depends on File['/etc/ntp.conf'] and end up at Package['ntp']. When Puppet looks for the dependencies

of Package['ntp'], it will end up back at File['/etc/ntp.conf'], forming a circular path. This type of problem is known as a circular dependency problem; Puppet can't decide where to start because the two resources depend on each other.

To fix the circular dependency problem, all you need to do is remove one of the dependency lines and break the circle. The following code fixes the problem:

Now when we run puppet apply or agent with the --graph option, the resulting graph does not have any circular paths (cycles):

How it works...

In this graph it is easy to see that Package[ntp] is the first resource to be applied, then File[/etc/ntp.conf], and finally Service[ntp].

There's more...

Resource See also

The Using run stages recipe in

Puppet's error messages can sometimes be a little confusing. Updated and increasingly helpful error messages are one reason to upgrade your Puppet installation if you are running any version prior to Version 3.

Here are some of the most common errors you might encounter, and what to do about them.

Often the first step is simply to search the Web for the error message text and see what explanations you can find for the error, along with any helpful advice about fixing it. Here are some of the most common puzzling errors, with possible explanations:

Where XXX is a file resource, you may have accidentally typed puppet://modules... in a file source instead of puppet:///modules... (note the triple slash):

The source file may not be present or may not be in the right location in the Puppet repo:

The file path may specify a parent directory (or directories) that doesn't exist. You can use separate file resources in Puppet to create these:

This is often caused by Puppet trying to write a file to a directory that doesn't exist. Check that the directory either exists already or is defined in Puppet, and that the file resource requires the directory (so that the directory is always created first):

This unhelpful error message is roughly translated as something went wrong. It tends to be a catch-all error caused by many different problems, but you may be able to determine what is wrong from the name of the resource, the class, or the module. One trick is to add the --debug switch, to get more useful information:

If you check your Git history to see what was touched in the most recent change, this may be another way to identify what's upsetting Puppet:

This can be caused by mistyping command line options, for example, if you type puppet -verbose instead of puppet --verbose. This kind of error can be hard to see:

This one has caused me a bit of puzzlement in the past. Puppet's complaining about a duplicate definition, and normally if you have two resources with the same name, Puppet will helpfully tell you where they are both defined. But in this case, it's indicating the same file and line number for both. How can one resource be a duplicate of itself?

The answer is, if it's a defined type (a resource created with the define keyword). If you create two instances of a defined type you'll also have two instances of all the resources contained within the definition, and they need to have distinct names. For example:

When we run Puppet, the same error is printed twice:

Because the exec resource is named is-process-running?, if you try to create more than one instance of the definition, Puppet will refuse because the result would be two exec resources with the same name. The solution is to include the name of the instance (or some other unique value) in the title of each resource:

Every resource must have a unique name, and a good way to ensure this with a definition is to interpolate the ${name} variable in its title. Note that we switched from using single to double quotes in the resource title:

The double quotes are required when you want Puppet to interpolate the value of a variable into a string.

How to do it...

Often the first step is simply to search the Web for the error message text and see what explanations you can find for the error, along with any helpful advice about fixing it. Here are some of the most common puzzling errors, with possible explanations:

Could not retrieve file metadata for XXX: getaddrinfo: Name or service not known

Where XXX is a file resource, you may have accidentally typed puppet://modules... in a file source instead of puppet:///modules... (note the triple slash):

Could not evaluate: Could not retrieve information from environment production source(s) XXX

The source file may not be present or may not be in the right location in the Puppet repo:

Error: Could not set 'file' on ensure: No such file or directory XXX

The file path may specify a parent directory (or directories) that doesn't exist. You can use separate file resources in Puppet to create these:

change from absent to file failed: Could not set 'file on ensure: No such file or directory

This is often caused by Puppet trying to write a file to a directory that doesn't exist. Check that the directory either exists already or is defined in Puppet, and that the file resource requires the directory (so that the directory is always created first):

undefined method 'closed?' for nil:NilClass

This unhelpful error message is roughly translated as something went wrong. It tends to be a catch-all error caused by many different problems, but you may be able to determine what is wrong from the name of the resource, the class, or the module. One trick is to add the --debug switch, to get more useful information:

[root@cookbook ~]# puppet agent -t --debug

If you check your Git history to see what was touched in the most recent change, this may be another way to identify what's upsetting Puppet:

Could not parse for environment --- "--- production": Syntax error at end of file at line 1

This can be caused by mistyping command line options, for example, if you type puppet -verbose instead of puppet --verbose. This kind of error can be hard to see:

Duplicate definition: X is already defined in [file] at line Y; cannot redefine at [file] line Y

This one has

caused me a bit of puzzlement in the past. Puppet's complaining about a duplicate definition, and normally if you have two resources with the same name, Puppet will helpfully tell you where they are both defined. But in this case, it's indicating the same file and line number for both. How can one resource be a duplicate of itself?

The answer is, if it's a defined type (a resource created with the define keyword). If you create two instances of a defined type you'll also have two instances of all the resources contained within the definition, and they need to have distinct names. For example:

When we run Puppet, the same error is printed twice:

Because the exec resource is named is-process-running?, if you try to create more than one instance of the definition, Puppet will refuse because the result would be two exec resources with the same name. The solution is to include the name of the instance (or some other unique value) in the title of each resource:

Every resource must have a unique name, and a good way to ensure this with a definition is to interpolate the ${name} variable in its title. Note that we switched from using single to double quotes in the resource title:

The double quotes are required when you want Puppet to interpolate the value of a variable into a string.

See also

The Generating reports recipe in this chapter
The Noop: the don't change anything option recipe in this chapter
The Logging debug messages recipe in this chapter

You probably know that Puppet's configuration settings are stored in puppet.conf, but there are many parameters, and those that aren't listed in puppet.conf will take a default value. How can you see the value of any configuration parameter, regardless of whether or not it's explicitly set in puppet.conf? The answer is to use the puppet config print command.

How to do it...

Run the following command. This will produce a lot of output (it may be helpful to pipe it through less if you'd like to browse the available configuration settings):

[root@cookbook ~]# puppet config print |head -25 report_serialization_format = pson hostcsr = /var/lib/puppet/ssl/csr_cookbook.example.com.pem filetimeout = 15 masterhttplog = /var/log/puppet/masterhttp.log pluginsignore = .svn CVS .git ldapclassattrs = puppetclass certdir = /var/lib/puppet/ssl/certs ignoreschedules = false disable_per_environment_manifest = false archive_files = false hiera_config = /etc/puppet/hiera.yaml req_bits = 4096 clientyamldir = /var/lib/puppet/client_yaml evaltrace = false module_working_dir = /var/lib/puppet/puppet-module tags = cacrl = /var/lib/puppet/ssl/ca/ca_crl.pem manifest = /etc/puppet/manifests/site.pp inventory_port = 8140 ignoreimport = false dbuser = puppet postrun_command = document_all = false splaylimit = 1800 certificate_expire_warning = 5184000 How it works...

Running puppet config print will See also

The Generating reports recipe in this chapter