In this section we will discuss five ways to extend and enhance Puppet in your environment. We'll show you how to review the changes Puppet has made, get a list of backups of files, use environments to test the changes, improve the Puppet server's performance, and use data from other applications in the Puppet policy.
The easiest way to review the history of changes made to the system is to examine the Puppet log. This is normally in the messages
log from syslog
, but you can also enable direct file logging in the puppet.conf
configuration. If you are just looking for the recent changes on a client system, the following is likely all you need:
$ grep puppet-agent /var/log/messages
You can also have the Puppet agent send a report to the server. The server can store or process these reports. Let's see how to set that up now. The first step is to enable the sending of reports to the server. Add the following line to /etc/puppet/puppet.conf
:
[agent] report = true
Now we need to tell the server what to do with the reports. Here is an example setting for the server:
[master] reports = tagmail, store
This tells the server to store the Puppet report and to give it to a report processor named tagmail
. You can also create your own report processor and list it here.
If you enable the store method, the puppet reports from each client node will be stored in a YAML format on the disk. Assuming that you used the paths recommended in this book, the directory /var/lib/puppet-server/reports/hostname
will contain one YAML file for each Puppet run. These will build up over time. If you enable this, you should also create a cron job to remove the older reports. Here's an example that Puppet can enforce:
cron { 'clean-up-reports': ensure => present, command => 'find /var/lib/puppet-server/reports –type f –name \*.yaml –mtime +21 –delete', user => puppet, weekday => 0, hour => 2, minute => 11, }
It is useful when starting to send reports to the tagmail
report processor. This allows you to send messages about changes to systems to people in e-mail. The message content can be based on either the log level of the message, or the tag applied to the resource. Here are some examples that I find useful. You define which messages should go to which e-mail addresses in the /etc/puppet/tagmail.conf
file. Following are some useful examples:
# Send every log message at every level to me all: [email protected] # Send critical messages to my pager alert,emergy,crit: [email protected] # Send log messages from the apache module to the webmaster apache,!info,!debug: [email protected] # Send every change notification or error to the operations team notice,warning,err,alert,emergy,crit: [email protected]
Note
The syntax may remind you of syslog.conf
; however, log levels are not inclusive of higher levels. If you put just info
as a tag, you will receive info messages, but nothing from notice
, alert
, critical
, and so on.
Tagmail is documented at http://docs.puppetlabs.com/references/latest/report.html#tagmail.
Puppet Dashboard is a product provided by Puppet Labs to store reports in a database and provides a web UI to review those changes. After you have installed and configured the Puppet Dashboard, you can enable it as a report processor in /etc/puppet/puppet.conf
:
[master] reports = tagmail, store, http reporturl = http://dashboard.example.net/reports/upload
At the time of printing of this book, the Puppet Labs documentation for installing Dashboard looks very intimidating. If you are using RHEL 6 or CentOS 6 as we suggested with the Puppet Labs repositories, you can safely ignore the Installing Dependancies section of the installation and just install the dashboard as follows:
$ sudo yum install mysql-server puppet-dashboard
Then perform the configuration steps documented at http://docs.puppetlabs.com/dashboard/manual/1.2/bootstrapping.html#configuring-dashboard.
At the time of printing, the page doesn't mention that the Puppet Labs-provided packages include the appropriate startup scripts for you. You don't need to create your own as the page suggests:
$ sudo /sbin/service puppet-dashboard start $ sudo /sbin/service puppet-dashboard-workers start $ sudo /sbin/chkconfig puppet-dashboard on $ sudo /sbin/chkconfig puppet-dashboard-workers on
Puppet Dashboard won't work very well unless you run it under Passenger. This is well documented in the manual. We will discuss this in the Passenger section.
You can build your own custom report processors to do anything you want with the provided data. The guidelines for writing your own custom report processors are available at http://docs.puppetlabs.com/guides/reporting.html.
When Puppet changes a file, it makes a backup of the file and stores it on the disk in the directory specified by the clientbucket
configuration option. You can pull the old versions of files out of this directory at any time. Puppet includes a command to restore the files, but it doesn't provide a command to find them. Given the odd directory structure with the file contents stored by their MD5 checksum, this isn't trivial. Here's a good script for finding the Puppet backups of a given file:
#!/bin/bash FILE=$1 FOUND=0 CLIENTBUCKET=$(puppet config print clientbucketdir) echo "Searching for local backups of $FILE..." for backup in $(find $CLIENTBUCKET -type f -name paths \ -exec grep -l $FILE {} \; |xargs -r ls -t); do hash=$( basename $(dirname $backup)) filename=$(< $backup ) modify_time=$(stat --format '%y' $backup) echo -e "$filename\t$hash\t$modify_time" FOUND=$((FOUND+1)) done if [ $FOUND -gt 0 ]; then if [ $FOUND -eq 1 ]; then echo "1 backup was found." elif [ $FOUND -gt 1 ]; then echo "$FOUND backups were found." fi echo "" echo "To view a file: puppet filebucket get -b $CLIENTBUCKET <hash>" echo "To restore a file: puppet filebucket restore -b $CLIENTBUCKET /new/path <hash>" else echo "No previous versions of the $FILE were found." fi
An exercise for the reader: use the file resource example from the Quick start section and replicate this script to all client nodes.
Environments provide you with the ability to isolate pieces of code and only invoke them for systems specifically configured for that environment, or only when the environment is enabled by hand. I'll show you how you can use environments to extend what Puppet can accomplish.
The following configuration option will provide flexibility when loading modules. This configuration will first check to see if a module exists in an environment-specific directory, then fall back to the main module directory. The rest of the following examples assume that you have added this option and restarted the Puppet master:
[master] modulepath = $confdir/env/$environment/modules:$confdir/modules
Once you are using puppet in production, you will want to avoid testing new modules on your production systems. Assuming you set the modulepath
as listed in previous configuration, you can put new the modules you want to test in /etc/puppet/env/testing/modules
. If you have a dedicated test system, you can put the following in the puppet.conf
:
[agent] environment = testing
You can also invoke the puppet agent in the testing environment on a case-by-case basis, if you need to test a new module on a particular system:
$ sudo puppet agent --test --environment testing
Using this approach, you can test out changes to your modules prior to putting the new version in the main module path and affecting hundreds of client systems.
Another way to use environments is for modules that you do not want to run every time, but only upon request. For example, you may have a module that pushes out a software release.
Assuming you set the modulepath
as listed before, you can put the new modules you want to test in /etc/puppet/env/release/modules
. To perform the software release, you would invoke Puppet on that system in the 'release' environment.
$ sudo puppet agent --test --environment release
The Puppet master process is a WEBrick ruby application server. It will handle only two concurrent Puppet clients. In any environment with more than 50 Puppet clients before it gets overloaded. If you intend to have more than 50 clients, you should disable the Puppet master process and enable it under a more powerful Ruby on the Rails application server. In this section we will document how to make it run under the popular Passenger Rails application server.
First, you will need to install the Apache web server and Phusion Passenger for your platform. The platform-specific instructions are available at http://www.modrails.com/documentation/Users%20guide%20Apache.html.
If you are using a Red Hat or CentOS system, it could be as simple as this:
$ sudo rpm –i http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm $ sudo rpm –i http://passenger.stealthymonkeys.com/rhel/6/passenger-release.noarch.rpm $ sudo yum install httpd rubygems mod_ssl mod_passenger
Configure the puppetmaster service as a Passenger application. We do this by creating a configuration file for Apache, which defines the Puppet master application, and a config.ru
file that contains the Phusion Passenger application environment. All of these steps must be done as root:
$ sudo bash # sed –e 's/squigley.namespace.at/puppet.example.net/' \ -e 's#/etc/puppet/ssl#/var/lib/puppet-server/ssl#' \ /usr/share/puppet/ext/rack/files/apache2.conf\ > /etc/httpd/conf.d/puppetmaster.conf # mkdir –p /etc/puppet/rack/public # sed –e 's#/var/lib/puppet#/var/lib/puppet-server#' \ /usr/share/puppet/ext/rack/files/config.ru \ >/etc/puppet/rack/config.ru # chown puppet /etc/puppet/rack/config.ru # exit
The chown
command shown the previous code snippet is essential. Passenger runs the application specified in the config.ru
file as the user who owns the file. Failing to set the owner of this file correctly is more than 50 percent of the issues we see on the mailing list.
Now, stop the Puppet master service (and Puppet Dashboard if you have it) and start Apache Passenger.
$ sudo /sbin/service puppetmaster stop Stopping puppetmaster: [OK] $ sudo /sbin/chkconfigpuppetmaster off $ sudo /sbin/chkconfighttpd on $ sudo /sbin/service httpd start Starting httpd: [OK]
At this point you can test your clients as before. Absolutely nothing should be different from a client/agent perspective.
To run Puppet Dashboard under Passenger, you would create a configuration file named /etc/httpd/conf.d/dashboard.conf
, similar to the example file /usr/share/puppet-dashboard/ext/passenger/dashboard-vhost.conf
. However, be aware that the first 12 lines duplicate the settings already present in the other files in this directory. I would start with the following file, and add more options from the example as necessary:
<VirtualHost *:80> ServerNamedashboard.example.net DocumentRoot /usr/share/puppet-dashboard/public/ RailsBaseURI / <Directory /usr/share/puppet-dashboard/public/> Options None Order allow,deny allow from all </Directory> ErrorLog /var/log/httpd/dashboard.example.net_error.log LogLevel warn CustomLog /var/log/httpd/dashboard.example.net_access.log combined ServerSignature On</VirtualHost>
Then perform the following steps to make the change active:
$ sudo /sbin/service puppet-dashboard stop Stopping Puppet Dashboard: [OK] $ sudo /sbin/chkconfig puppet-dashboard off $ sudo rm /etc/httpd/conf.d/welcome.conf $ sudo /sbin/service httpd restart Stopping httpd: [OK] Starting httpd: [OK]
Do you already have a source of data about your hosts or users? You really wouldn't want to maintain that same data in your Puppet policy now, would you? In this section we will talk about how to import external data and use it in your Puppet policy.
The best way to source external data is through a component called Hiera. Since you can import any kind of data, this topic is worthy of its own book. But we will set up a very small but powerful example that you can easily expand on: providing a list of user accounts.
All examples in this section use the YAML data format. You can install modules to provide other data backends, like JSON or MySQL. Puppet 3 includes only two Hiera backends by default: YAML and Puppet. The Puppet backend allows you to lookup data from Puppet and return it with the YAML data in a single result set.
First, we need to tell Puppet where to find Hiera data for this class. Assuming we would like to store the Heira input data in /etc/puppet/hiera
, you should create the file /etc/puppet/hiera.yaml
with the following input. Note that YAML is very exact in its syntax, so you need to type in every dash and every space in this text:
--- :backends: - yaml :yaml: :datadir: '/etc/puppet/hiera' :hierarchy: - %{::hostname} - common
This file tells Puppet that the Hiera data format will be YAML, and the files will be in the /etc/puppet/hiera
directory. Then it tells Puppet to first look for a file with the name of the client host (from the hostname
fact) with .yaml
appended to the end, then to look for a file named common.yaml
.
Next we should create the actual YAML data. Here is an example we could use for /etc/puppet/hiera/common.yaml
:
--- managehomedirs: true users: root: uid: 0 gid: 0 jack: uid: 1002 gid: 1002 jill: uid: 1001 gid: 1001
If we wanted to add the user john
to the system named web1
, we would create another file /etc/puppet/hiera/web1.yaml
:
--- users: john: uid: 1003 gid: 1003
Note
You might be asking yourself: Isn't this just another file I have to maintain? Yes, in our example, this file is a static text file. In real life, you would have this data be output in the YAML or JSON format from your other application. You can also create a custom Hiera backend to query the application directly for the data.
Next we use a function to import data from Hiera into the Puppet module. Back in our theoretical users
module, the import functions would look as follows:
$managehomedirs = hiera('managehomedirs') $users = hiera_hash('users')
The first function there will find a single value for managehomedirs
. With the hierarchy we named in the preceding example, it will look first for a YAML file named for the hostname, then it will look in common.yaml
. Since this value is defined is in the common file, managehomedirs
will be set to true
.
The second Hiera function will return a merged hash of every value for that name in the hierarchy. For any system other than web1
, the array will contain root
, jack
, and jill
. For the web1
system it will contain john
, root
, jack
, and jill
.
Now, here is a complete module for managing the users on your system. You'll note that this module includes no data. All data is sourced from Hiera, thus allowing Puppet to use another application as a source for policy data:
class 'users' { $managehomedirs = hiera('managehomedirs',true) $defaults = { managehome => $managehomedirs } $userlist = hiera_hash('users') create_resources( user, $userlist, $defaults ) }
This module is pretty straightforward:
We load in the value for
managehomedirs
. If we cannot find the value, we default totrue
. We then use this as the sole key in a hash of default values.We load in every user defined in every backend in the hierarchy.
We invoke the
createresources()
function to create a user resource for each user defined in our previous Hiera data files.
Note
This is a very powerful module. If you have a thousand user entries in your YAML data, it will create a thousand user resources in the Puppet catalog.
This section has provided you with a quick snapshot into the power and flexibility provided by Hiera data lookups in Puppet. You can create extensive hierarchies for data lookup, and you can use multiple data backends to provide policy data to Puppet. This will allow you to create Puppet policies which are independent of the data and source all of your configuration data from another application you use.
You can find more information about using Hiera at http://puppetlabs.com/blog/first-look-installing-and-using-hiera/.
You can find more information about the YAML data format at http://www.yaml.org/.