Book Image

Troubleshooting Puppet

By : Thomas Uphill
Book Image

Troubleshooting Puppet

By: Thomas Uphill

Overview of this book

Table of Contents (14 chapters)

Log files and the catalog


Logging in Puppet can be enabled on a client node (agent) with the --debug option to Puppet agent. This will output a lot of information. Each plugin file will be displayed as it is being read and executed. Once the catalog compiles, as each resource is applied to the machine, debugging information will be shown on the agent.

However, when you are debugging, your catalog may fail to compile. If this is the case, then you will need to examine the logs on the master. Where the logs are kept on the master depends on the way you have your master configured. The Puppet master process can be run either through a Ruby HTTP library named WEBrick, or via Passenger on a web server such as Apache or Nginx. Also, a third option now exists. You can also use the puppetserver application, which is a combination of JRuby and Clojure.

puppet master

Both the WEBrick and Passenger methods of running a Puppet master are equivalent to running puppet master from the command line. The configuration options for the Puppet master can be viewed with puppet help master.

By default, Puppet will log using syslog to the system logs (usually /var/log/messages). You can change this by making the --logdest option point at a file (logdest is used to specify the destination for log files, logdest may one of syslog, console, or the path to a file). If you are running the WEBrick server, then you can start the server like this:

# puppet master --logdest /var/log/puppet/master.log

Tip

When using Passenger, you will have a config.ru file, which is installed with the puppet-passenger package. You can add the additional logging options to this file.

To enable the debugging of logs, add the --debug option in addition to the --logdest option. You may also enable the --verbose option.

puppetserver

puppetserver is the new server for Puppet that is based on the server for PuppetDB. It uses a Java Virtual Machine (JVM) to run JRuby for the Puppet Server. This mechanism also uses Clojure. Puppet Labs has already made puppetserver the default Puppet master implementation for the new installations of Puppet Enterprise. The configuration of puppetserver is different from the Puppet master configuration. The server is configured by files that are located in /etc/puppetserver by default. Since the server is running through a JVM, it uses the Logback library. The configuration for Logback is in /etc/puppetserver/logback.xml. To enable debug logs, edit this file and change the log level from info to debug, as follows:

<root level="debug">
<!--<appender-ref ref="STDOUT"/>-->
<appender-ref ref="${logappender:-DUMMY}" />
<appender-ref ref="F1"/>
</root>

Changes made to this file are recognized immediately by the server. There is no need to restart the service. For entirely too much information, try setting the level to trace. More information on puppetserver can be found at https://github.com/puppetlabs/puppet-server.

jq

When the catalog is compiled for a node, the master will send the catalog to the node. The node will store the catalog in the client_data subdirectory of /var/lib/puppet. The catalog will be in the JSON format (previous versions of Puppet used the YAML format for catalogs). Reading the JSON files is not as simple as reading the YAML files. A tool that can help make things easier is jq, a command-line JSON processor. You can use it to search through the JSON files. For instance, to view the classes within a catalog, use the following:

$ jq '.data.classes[]' <hostname.json
"settings"
"default"

To view the resources defined in the catalog, use the following command:

$ jq '.data.resources[]'<hostname.json

To filter out the resources tagged with a specific tag, use "class" as follows:

jq '.data.resources[] | select(.tags[] == "class")' <hostname.json

Once you learn the syntax for jq, searching through large catalogs becomes an easy task. This can help you find out where your resources are defined quickly. More information on jq can be found at http://stedolan.github.io/jq/.