Book Image

DevOps: Puppet, Docker, and Kubernetes

By : Neependra Khare, Ke-Jou Carol Hsu, Hideto Saito, John Arundel, Hui-Chuan Chloe Lee, Thomas Uphill
Book Image

DevOps: Puppet, Docker, and Kubernetes

By: Neependra Khare, Ke-Jou Carol Hsu, Hideto Saito, John Arundel, Hui-Chuan Chloe Lee, Thomas Uphill

Overview of this book

With so many IT management and DevOps tools on the market, both open source and commercial, it’s difficult to know where to start. DevOps is incredibly powerful when implemented correctly, and here’s how to get it done.This Learning Path covers three broad areas: Puppet, Docker, and Kubernetes. This Learning Path is a large resource of recipes to ease your daily DevOps tasks. We begin with recipes that help you develop a complete and expert understanding of Puppet’s latest and most advanced features. Then we provide recipes that help you efficiently work with the Docker environment. Finally, we show you how to better manage containers in different scenarios in production using Kubernetes. This course is based on these books: 1. Puppet Cookbook, Third Edition 2. Docker Cookbook 3. Kubernetes Cookbook

Related Content you might be interested in

Current Title:

Small book image
DevOps: Puppet, Docker, and Kubernetes
Neependra Khare | Ke-Jou Carol Hsu | Hideto Saito | John Arundel | Hui-Chuan Chloe Lee | Thomas Uphill
Mar, 2017 | 15 hours 25 minutes
No titles found
Table of Contents (6 chapters)
Preface
Preface
What this learning path covers
What you need for this learning path
Who this learning path is for
Reader feedback
Customer support
Free Chapter
1
1. Module 1
1. Module 1
1. Puppet Language and Style
2. Puppet Infrastructure
3. Writing Better Manifests
4. Working with Files and Packages
5. Users and Virtual Resources
6. Managing Resources and Files
7. Managing Applications
8. Internode Coordination
9. External Tools and the Puppet Ecosystem
10. Monitoring, Reporting, and Troubleshooting
2
2. Module 2
2. Module 2
1. Introduction and Installation
2. Working with Docker Containers
3. Working with Docker Images
4. Network and Data Management for Containers
5. Docker Use Cases
6. Docker APIs and Language Bindings
7. Docker Performance
8. Docker Orchestration and Hosting Platforms
9. Docker Security
10. Getting Help and Tips and Tricks
3
3. Module 3
3. Module 3
1. Building Your Own Kubernetes
2. Walking through Kubernetes Concepts
3. Playing with Containers
4. Building a High Availability Cluster
5. Building a Continuous Delivery Pipeline
6. Building Kubernetes on AWS
7. Advanced Cluster Administration
8. Logging and Monitoring
4
B. Bibliography
B. Bibliography
5
Index
Index

"Nothing is a problem, until it's a problem."

In this chapter, we will cover the following recipes:

Users can be a real pain. I don't mean the people, though doubtless that's sometimes true. But keeping UNIX user accounts and file permissions in sync across a network of machines, some of them running different operating systems, can be very challenging without some kind of centralized configuration management.

Each new developer who joins the organization needs an account on every machine, along with sudo privileges and group memberships, and needs their SSH key authorized for a bunch of different accounts. The system administrator who has to take care of this manually will be at the job all day, while the system administrator who uses Puppet will be done in minutes, and head out for an early lunch.

In this chapter, we'll look at some handy patterns and techniques to manage users and their associated resources. Users are also one of the most common applications for virtual resources, so we'll find out all about those. In the final section, we'll introduce exported resources, which are related to virtual resources.

Virtual resources in Puppet might seem complicated and confusing but, in fact, they're very simple. They're exactly like regular resources, but they don't actually take effect until they're realized (in the sense of "made real"); whereas a regular resource can only be declared once per node (so two classes can't declare the same resource, for example). A virtual resource can be realized as many times as you like.

This comes in handy when you need to move applications and services between machines. If two applications that use the same resource end up sharing a machine, they would cause a conflict unless you make the resource virtual.

To clarify this, let's look at a typical situation where virtual resources might come in handy.

You are responsible for two popular web applications: WordPress and Drupal. Both are web apps running on Apache, so they both require the Apache package to be installed. The definition for WordPress might look something like the following:

class wordpress {
  package {'httpd':
    ensure => 'installed',
  }
  service {'httpd':
    ensure => 'running',
    enable => true,
  }
}

The definition for Drupal might look like this:

class drupal {
  package {'httpd':
    ensure => 'installed',
  }
  service {'httpd':
    ensure => 'running',
    enable => true,
  }
}

All is well until you need to consolidate both apps onto a single server:

node 'bigbox' {
  include wordpress
  include drupal
}

Now Puppet will complain because you tried to define two resources with the same name: httpd.

Using virtual resources

You could remove the duplicate Apache package definition from one of the classes, but then nodes without the class including Apache would fail. You can get around this problem by putting the Apache package in its own class and then using include apache everywhere it's needed; Puppet doesn't mind you including the same class multiple times. In reality, putting Apache in its own class solves most problems but, in general, this method has the disadvantage that every potentially conflicting resource must have its own class.

Virtual resources can be used to solve this problem. A virtual resource is just like a normal resource, except that it starts with an @ character:

@package { 'httpd': ensure => installed }

You can think of it as being like a placeholder resource; you want to define it but you aren't sure you are going to use it yet. Puppet will read and remember virtual resource definitions, but won't actually create the resource until you realize the resource.

To create the resource, use the realize function:

realize(Package['httpd'])

You can call realize as many times as you want on the resource and it won't result in a conflict. So virtual resources are the way to go when several different classes all require the same resource, and they may need to coexist on the same node.

Here's how to build the example using virtual resources:

  1. Create the virtual module with the following contents:
    class virtual {
  @package {'httpd': ensure => installed }
  @service {'httpd': 
    ensure  => running,
    enable  => true,
    require => Package['httpd']
  }
}
  2. Create the Drupal module with the following contents:
    class drupal {
  include virtual
  realize(Package['httpd'])
  realize(Service['httpd'])
}
  3. Create the WordPress module with the following contents:
    class wordpress {
  include virtual
  realize(Package['httpd'])
  realize(Service['httpd'])
}
  4. Modify your site.pp file as follows:
    node 'bigbox' {
  include drupal
  include wordpress
}
  5. Run Puppet:
    bigbox# puppet agent -t
Info: Caching catalog for bigbox.example.com
Info: Applying configuration version '1413179615'
Notice: /Stage[main]/Virtual/Package[httpd]/ensure: created
Notice: /Stage[main]/Virtual/Service[httpd]/ensure: ensure changed 'stopped' to 'running'
Info: /Stage[main]/Virtual/Service[httpd]: Unscheduling refresh on Service[httpd]
Notice: Finished catalog run in 6.67 seconds

You define the package and service as virtual resources in one place: the virtual class. All nodes can include this class and you can put all your virtual services and packages in it. None of the packages will actually be installed on a node or services started until you call realize:

class virtual {
  @package { 'httpd': ensure => installed }
}

Every class that needs the Apache package can call realize on this virtual resource:

class drupal {
  include virtual
  realize(Package['httpd'])
}

Puppet knows, because you made the resource virtual, that you intended to have multiple references to the same package, and didn't just accidentally create two resources with the same name. So it does the right thing.

To realize virtual resources, you can also use the collection spaceship syntax:

Package <| title = 'httpd' |>

The advantage of this syntax is that you're not restricted to the resource name; you could also use a tag, for example:

Package <| tag = 'web' |>

Alternatively, you can just specify all instances of the resource type, by leaving the query section blank:

Package <| |>
How to do it...

Here's how to build the example using virtual resources:

Create the virtual module with the following contents:
class virtual {
  @package {'httpd': ensure => installed }
  @service {'httpd': 
    ensure  => running,
    enable  => true,
    require => Package['httpd']
  }
}
Create the Drupal module with the following contents:
class drupal {
  include virtual
  realize(Package['httpd'])
  realize(Service['httpd'])
}
Create the WordPress module with the following contents:
class wordpress {
  include virtual
  realize(Package['httpd'])
  realize(Service['httpd'])
}
Modify your site.pp file as follows:
node 'bigbox' {
  include drupal
  include wordpress
}
Run
  1. Puppet:
    bigbox# puppet agent -t
Info: Caching catalog for bigbox.example.com
Info: Applying configuration version '1413179615'
Notice: /Stage[main]/Virtual/Package[httpd]/ensure: created
Notice: /Stage[main]/Virtual/Service[httpd]/ensure: ensure changed 'stopped' to 'running'
Info: /Stage[main]/Virtual/Service[httpd]: Unscheduling refresh on Service[httpd]
Notice: Finished catalog run in 6.67 seconds

You define the package and service as virtual resources in one place: the virtual class. All nodes can include this class and you can put all your virtual services and packages in it. None of the packages will actually be installed on a node or services started until you call realize:

class virtual {
  @package { 'httpd': ensure => installed }
}

Every class that needs the Apache package can call realize on this virtual resource:

class drupal {
  include virtual
  realize(Package['httpd'])
}

Puppet knows, because you made the resource virtual, that you intended to have multiple references to the same package, and didn't just accidentally create two resources with the same name. So it does the right thing.

To realize virtual resources, you can also use the collection spaceship syntax:

Package <| title = 'httpd' |>

The advantage of this syntax is that you're not restricted to the resource name; you could also use a tag, for example:

Package <| tag = 'web' |>

Alternatively, you can just specify all instances of the resource type, by leaving the query section blank:

Package <| |>
How it works...

You define the package and service as virtual resources in one place: the virtual class. All nodes can include this class and you can put all your virtual services and packages in it. None of the packages will actually be installed on a node or services started until you call realize: 


class virtual {
@package { 'httpd': ensure => installed }
}


Every class that needs the Apache package can call realize on this virtual resource:


class drupal {
include virtual
realize(Package['httpd'])
}


Puppet knows, because you 

made the resource virtual, that you intended to have multiple references to the same package, and didn't just accidentally create two resources with the same name. So it does the right thing.
To realize virtual resources, you can also use the collection spaceship syntax:
Package <| title = 'httpd' |>
The advantage of this syntax is that you're not restricted to the resource name; you could also use a tag, for example:
Package <| tag = 'web' |>
Alternatively, you can just specify all instances of the resource type, by leaving the query section blank:
Package <| |>


There's more...


To realize 

virtual resources, you can also use the collection spaceship syntax:
Package <| title = 'httpd' |>
The advantage of this syntax is that you're not restricted to the resource name; you could also use a tag, for example:
Package <| tag = 'web' |>
Alternatively, you can just specify all instances of the resource type, by leaving the query section blank:
Package <| |>



Users are a great example of a resource that may need to be realized by multiple classes. Consider the following situation. To simplify administration of a large number of machines, you defined classes for two kinds of users: developers and sysadmins. All machines need to include sysadmins, but only some machines need developers:
node 'server' { 
  include user::sysadmins 
}

node 'webserver' {
  include user::sysadmins 
  include user::developers 
}
However, some users may be members of both groups. If each group simply declares its members as regular user resources, this will lead to a conflict when a node includes both developers and sysadmins, as in the webserver example.
To avoid this conflict, a common pattern is to make all users virtual resources, defined in a single class user::virtual that every machine includes, and then realizing the users where they are needed. This way, there will be no conflict if a user is a member of multiple groups.
Follow these steps to create a user::virtual class:
  1. Create the file modules/user/manifests/virtual.pp with the following contents:
    class user::virtual {
  @user { 'thomas':  ensure => present }
  @user { 'theresa': ensure => present }
  @user { 'josko':   ensure => present }
  @user { 'nate':    ensure => present }
}
  2. Create the file modules/user/manifests/developers.pp with the following contents:
    class user::developers {
  realize(User['theresa'])
  realize(User['nate'])
}
  3. Create the file modules/user/manifests/sysadmins.pp with the following contents:
    class user::sysadmins {
  realize(User['thomas'])
  realize(User['theresa'])
  realize(User['josko'])
}
  4. Modify your nodes.pp file as follows:
    node 'cookbook' {
  include user::virtual
  include user::sysadmins
  include user::developers
}
  5. Run Puppet:
    cookbook# puppet agent -t
Info: Caching catalog for cookbook.example.com
Info: Applying configuration version '1413180590'
Notice: /Stage[main]/User::Virtual/User[theresa]/ensure: created
Notice: /Stage[main]/User::Virtual/User[nate]/ensure: created
Notice: /Stage[main]/User::Virtual/User[thomas]/ensure: created
Notice: /Stage[main]/User::Virtual/User[josko]/ensure: created
Notice: Finished catalog run in 0.47 seconds

When we include the user::virtual class, all the users are declared as virtual resources (because we included the @ symbol):
  @user { 'thomas':  ensure => present }
  @user { 'theresa': ensure => present }
  @user { 'josko':   ensure => present }
  @user { 'nate':    ensure => present }
That is to say, the resources exist in Puppet's catalog; they can be referred to by and linked with other resources, and they are in every respect identical to regular resources, except that Puppet doesn't actually create the corresponding users on the machine.
In order for that to happen, we need to call realize on the virtual resources. When we include the user::sysadmins class, we get the following code:
  realize(User['thomas'])
  realize(User['theresa'])
  realize(User['josko'])
Calling realize on a virtual resource tells Puppet, "I'd like to use that resource now". This is what it does, as we can see from the run output:
Notice: /Stage[main]/User::Virtual/User[theresa]/ensure: created

However, Theresa is in both the developers and sysadmins classes! Won't that mean we end up calling realize twice on the same resource?
realize(User['theresa'])
...
realize(User['theresa'])
Yes, it does, and that's fine. You're explicitly allowed to realize resources multiple times, and there will be no conflict. So long as some class, somewhere, calls realize on Theresa's account, it will be created. Unrealized resources are simply discarded during catalog compilation.
When you use this pattern to manage your own users, every node should include the user::virtual class, as a part of your basic housekeeping configuration. This class will declare all users (as virtual) in your organization or site. This should also include any users who exist only to run applications or services (such as Apache, www-data, or deploy, for example). Then, you can realize them as needed on individual nodes or in specific classes.
For production use, you'll probably also want to specify a UID and GID for each user or group, so that these numeric identifiers are synchronized across your network. You can do this using the uid and gid parameters for the user resource.
Note
If you don't specify a user's UID, for example, you'll just get whatever is the next ID number available on a given machine, so the same user on different machines will have a different UID. This can lead to permission problems when using shared storage, or moving files between machines.
A common pattern when defining users as virtual resources is to assign tags to the users based on their assigned roles within your organization. You can then use the collector syntax instead of realize to collect users with specific tags applied.
For example, see the following code snippet:
@user { 'thomas':  ensure => present, tag => 'sysadmin' }
@user { 'theresa': ensure => present, tag => 'sysadmin' }
@user { 'josko':   ensure => present, tag => 'dev' }
User <| tag == 'sysadmin' |>
In the previous example, only users thomas and theresa would be included.


How to do it...


Follow these steps

 to create a user::virtual class:
  1. Create the file modules/user/manifests/virtual.pp with the following contents:
    class user::virtual {
  @user { 'thomas':  ensure => present }
  @user { 'theresa': ensure => present }
  @user { 'josko':   ensure => present }
  @user { 'nate':    ensure => present }
}
  2. Create the file modules/user/manifests/developers.pp with the following contents:
    class user::developers {
  realize(User['theresa'])
  realize(User['nate'])
}
  3. Create the file modules/user/manifests/sysadmins.pp with the following contents:
    class user::sysadmins {
  realize(User['thomas'])
  realize(User['theresa'])
  realize(User['josko'])
}
  4. Modify your nodes.pp file as follows:
    node 'cookbook' {
  include user::virtual
  include user::sysadmins
  include user::developers
}
  5. Run Puppet:
    cookbook# puppet agent -t
Info: Caching catalog for cookbook.example.com
Info: Applying configuration version '1413180590'
Notice: /Stage[main]/User::Virtual/User[theresa]/ensure: created
Notice: /Stage[main]/User::Virtual/User[nate]/ensure: created
Notice: /Stage[main]/User::Virtual/User[thomas]/ensure: created
Notice: /Stage[main]/User::Virtual/User[josko]/ensure: created
Notice: Finished catalog run in 0.47 seconds

When we include the user::virtual class, all the users are declared as virtual resources (because we included the @ symbol):
  @user { 'thomas':  ensure => present }
  @user { 'theresa': ensure => present }
  @user { 'josko':   ensure => present }
  @user { 'nate':    ensure => present }
That is to say, the resources exist in Puppet's catalog; they can be referred to by and linked with other resources, and they are in every respect identical to regular resources, except that Puppet doesn't actually create the corresponding users on the machine.
In order for that to happen, we need to call realize on the virtual resources. When we include the user::sysadmins class, we get the following code:
  realize(User['thomas'])
  realize(User['theresa'])
  realize(User['josko'])
Calling realize on a virtual resource tells Puppet, "I'd like to use that resource now". This is what it does, as we can see from the run output:
Notice: /Stage[main]/User::Virtual/User[theresa]/ensure: created

However, Theresa is in both the developers and sysadmins classes! Won't that mean we end up calling realize twice on the same resource?
realize(User['theresa'])
...
realize(User['theresa'])
Yes, it does, and that's fine. You're explicitly allowed to realize resources multiple times, and there will be no conflict. So long as some class, somewhere, calls realize on Theresa's account, it will be created. Unrealized resources are simply discarded during catalog compilation.
When you use this pattern to manage your own users, every node should include the user::virtual class, as a part of your basic housekeeping configuration. This class will declare all users (as virtual) in your organization or site. This should also include any users who exist only to run applications or services (such as Apache, www-data, or deploy, for example). Then, you can realize them as needed on individual nodes or in specific classes.
For production use, you'll probably also want to specify a UID and GID for each user or group, so that these numeric identifiers are synchronized across your network. You can do this using the uid and gid parameters for the user resource.
Note
If you don't specify a user's UID, for example, you'll just get whatever is the next ID number available on a given machine, so the same user on different machines will have a different UID. This can lead to permission problems when using shared storage, or moving files between machines.
A common pattern when defining users as virtual resources is to assign tags to the users based on their assigned roles within your organization. You can then use the collector syntax instead of realize to collect users with specific tags applied.
For example, see the following code snippet:
@user { 'thomas':  ensure => present, tag => 'sysadmin' }
@user { 'theresa': ensure => present, tag => 'sysadmin' }
@user { 'josko':   ensure => present, tag => 'dev' }
User <| tag == 'sysadmin' |>
In the previous example, only users thomas and theresa would be included.


How it works...


When we include

 the user::virtual class, all the users are declared as virtual resources (because we included the @ symbol):
  @user { 'thomas':  ensure => present }
  @user { 'theresa': ensure => present }
  @user { 'josko':   ensure => present }
  @user { 'nate':    ensure => present }
That is to say, the resources exist in Puppet's catalog; they can be referred to by and linked with other resources, and they are in every respect identical to regular resources, except that Puppet doesn't actually create the corresponding users on the machine.
In order for that to happen, we need to call realize on the virtual resources. When we include the user::sysadmins class, we get the following code:
  realize(User['thomas'])
  realize(User['theresa'])
  realize(User['josko'])
Calling realize on a virtual resource tells Puppet, "I'd like to use that resource now". This is what it does, as we can see from the run output:
Notice: /Stage[main]/User::Virtual/User[theresa]/ensure: created

However, Theresa is in both the developers and sysadmins classes! Won't that mean we end up calling realize twice on the same resource?
realize(User['theresa'])
...
realize(User['theresa'])
Yes, it does, and that's fine. You're explicitly allowed to realize resources multiple times, and there will be no conflict. So long as some class, somewhere, calls realize on Theresa's account, it will be created. Unrealized resources are simply discarded during catalog compilation.
When you use this pattern to manage your own users, every node should include the user::virtual class, as a part of your basic housekeeping configuration. This class will declare all users (as virtual) in your organization or site. This should also include any users who exist only to run applications or services (such as Apache, www-data, or deploy, for example). Then, you can realize them as needed on individual nodes or in specific classes.
For production use, you'll probably also want to specify a UID and GID for each user or group, so that these numeric identifiers are synchronized across your network. You can do this using the uid and gid parameters for the user resource.
Note
If you don't specify a user's UID, for example, you'll just get whatever is the next ID number available on a given machine, so the same user on different machines will have a different UID. This can lead to permission problems when using shared storage, or moving files between machines.
A common pattern when defining users as virtual resources is to assign tags to the users based on their assigned roles within your organization. You can then use the collector syntax instead of realize to collect users with specific tags applied.
For example, see the following code snippet:
@user { 'thomas':  ensure => present, tag => 'sysadmin' }
@user { 'theresa': ensure => present, tag => 'sysadmin' }
@user { 'josko':   ensure => present, tag => 'dev' }
User <| tag == 'sysadmin' |>
In the previous example, only users thomas and theresa would be included.


There's more...


When you use this 

pattern to manage your own users, every node should include the user::virtual class, as a part of your basic housekeeping configuration. This class will declare all users (as virtual) in your organization or site. This should also include any users who exist only to run applications or services (such as Apache, www-data, or deploy, for example). Then, you can realize them as needed on individual nodes or in specific classes.
For production use, you'll probably also want to specify a UID and GID for each user or group, so that these numeric identifiers are synchronized across your network. You can do this using the uid and gid parameters for the user resource.
Note
If you don't specify a user's UID, for example, you'll just get whatever is the next ID number available on a given machine, so the same user on different machines will have a different UID. This can lead to permission problems when using shared storage, or moving files between machines.
A common pattern when defining users as virtual resources is to assign tags to the users based on their assigned roles within your organization. You can then use the collector syntax instead of realize to collect users with specific tags applied.
For example, see the following code snippet:
@user { 'thomas':  ensure => present, tag => 'sysadmin' }
@user { 'theresa': ensure => present, tag => 'sysadmin' }
@user { 'josko':   ensure => present, tag => 'dev' }
User <| tag == 'sysadmin' |>
In the previous example, only users thomas and theresa would be included.


See also


The Using virtual resources recipe in this chapter


The Managing users' customization files recipe in this chapter





A sensible approach to access control for servers is to use named user accounts with passphrase-protected SSH keys, rather than having users share an account with a widely known password. Puppet makes this easy to manage thanks to the built-in ssh_authorized_key type.
To combine this with virtual users, as described in the previous section, you can create a define, which includes both the user and ssh_authorized_key resources. This will also come in handy when adding customization files and other resources to each user.
Follow these steps to extend your virtual users' class to include SSH access:
  1. Create a new module ssh_user to contain our ssh_user definition. Create the modules/ssh_user/manifests/init.pp file as follows:
    define ssh_user($key,$keytype) {
  user { $name:
    ensure     => present,
  }

  file { "/home/${name}":
    ensure => directory,
    mode   => '0700',
    owner  => $name,
    require => User["$name"]
  }
  file { "/home/${name}/.ssh":
    ensure => directory,
    mode   => '0700',
    owner  => "$name",
    require => File["/home/${name}"],
  }

  ssh_authorized_key { "${name}_key":
    key     => $key,
    type    => "$keytype",
    user    => $name,
    require => File["/home/${name}/.ssh"],
  }
}
  2. Modify your modules/user/manifests/virtual.pp file, comment out the previous definition for user thomas, and replace it with the following:
    @ssh_user { 'thomas':
  key     => 'AAAAB3NzaC1yc2E...XaWM5sX0z',
  keytype => 'ssh-rsa'
}
  3. Modify your modules/user/manifests/sysadmins.pp file as follows:
    class user::sysadmins {
    realize(Ssh_user['thomas'])
}
  4. Modify your site.pp file as follows:
    node 'cookbook' {
  include user::virtual
  include user::sysadmins
}
  5. Run Puppet:
    cookbook# puppet agent -t
Info: Caching catalog for cookbook.example.com
Info: Applying configuration version '1413254461'
Notice: /Stage[main]/User::Virtual/Ssh_user[thomas]/File[/home/thomas/.ssh]/ensure: created
Notice: /Stage[main]/User::Virtual/Ssh_user[thomas]/Ssh_authorized_key[thomas_key]/ensure: created
Notice: Finished catalog run in 0.11 seconds

For each user in our user::virtual class, we need to create:
We could declare separate resources to implement all of these for each user, but it's much easier to create a definition instead, which wraps them into a single resource. By creating a new module for our definition, we can refer to ssh_user from anywhere (in any scope):
define ssh_user ($key, $keytype) { 
  user { $name:
    ensure     => present,
  }
After we create the user, we can then create the home directory; we need the user first so that when we assign ownership, we can use the username, owner => $name:
  file { "/home/${name}":
    ensure => directory,
    mode => '0700',
    owner => $name,
    require => User["$name"]
  }
Tip
Puppet can create the users' home directory using the managehome attribute to the user resource. Relying on this mechanism is problematic in practice, as it does not account for users that were created outside of Puppet without home directories.
Next, we need to ensure that the .ssh directory exists within the home directory of the user. We require the home directory, File["/home/${name}"], since that needs to exist before we create this subdirectory. This implies that the user already exists because the home directory required the user:
  file { "/home/${name}/.ssh":
    ensure => directory,
    mode   => '0700',
    owner  => $name ,
    require => File["/home/${name}"],
  }
Finally, we create the ssh_authorized_key resource, again requiring the containing folder (File["/home/${name}/.ssh"]). We use the $key and $keytype variables to assign the key and type parameters to the ssh_authorized_key type as follows:
  ssh_authorized_key { "${name}_key":
    key     => $key,
    type    => "$keytype",
    user    => $name,
    require => File["/home/${name}/.ssh"],
  }
}
We passed the $key and $keytype variables when we defined the ssh_user resource for thomas:
@ssh_user { 'thomas':
  key => 'AAAAB3NzaC1yc2E...XaWM5sX0z',
  keytype => 'ssh-rsa'
}
Tip
The value for key, in the preceding code snippet, is the ssh key's public key value; it is usually stored in an id_rsa.pub file.
Now, with everything defined, we just need to call realize on thomas for all these resources to take effect:
realize(Ssh_user['thomas'])
Notice that this time the virtual resource we're realizing is not simply the user resource, as before, but the ssh_user defined type we created, which includes the user and the related resources needed to set up the SSH access:
Notice: /Stage[main]/User::Virtual/Ssh_user[thomas]/User[thomas]/ensure: created
Notice: /Stage[main]/User::Virtual/Ssh_user[thomas]/File[/home/thomas]/ensure: created
Notice: /Stage[main]/User::Virtual/Ssh_user[thomas]/File[/home/thomas/.ssh]/ensure: created
Notice: /Stage[main]/User::Virtual/Ssh_user[thomas]/Ssh_authorized_key[thomas_key]/ensure: created

Of course, you can add whatever resources you like to the ssh_user definition to have Puppet automatically create them for new users. We'll see an example of this in the next recipe, Managing users' customization files.


How to do it...


Follow these steps to extend your virtual users' class to include SSH access:


Create a new module ssh_user to contain our ssh_user definition. Create the modules/ssh_user/manifests/init.pp file as follows:
define ssh_user($key,$keytype) {
  user { $name:
    ensure     => present,
  }

  file { "/home/${name}":
    ensure => directory,
    mode   => '0700',
    owner  => $name,
    require => User["$name"]
  }
  file { "/home/${name}/.ssh":
    ensure => directory,
    mode   => '0700',
    owner  => "$name",
    require => File["/home/${name}"],
  }

  ssh_authorized_key { "${name}_key":
    key     => $key,
    type    => "$keytype",
    user    => $name,
    require => File["/home/${name}/.ssh"],
  }
}



Modify your modules/user/manifests/virtual.pp file, comment out the previous definition for user thomas, and replace it with the following:
@ssh_user { 'thomas':
  key     => 'AAAAB3NzaC1yc2E...XaWM5sX0z',
  keytype => 'ssh-rsa'
}



Modify 

  1. your modules/user/manifests/sysadmins.pp file as follows:
    class user::sysadmins {
    realize(Ssh_user['thomas'])
}
  2. Modify your site.pp file as follows:
    node 'cookbook' {
  include user::virtual
  include user::sysadmins
}
  3. Run Puppet:
    cookbook# puppet agent -t
Info: Caching catalog for cookbook.example.com
Info: Applying configuration version '1413254461'
Notice: /Stage[main]/User::Virtual/Ssh_user[thomas]/File[/home/thomas/.ssh]/ensure: created
Notice: /Stage[main]/User::Virtual/Ssh_user[thomas]/Ssh_authorized_key[thomas_key]/ensure: created
Notice: Finished catalog run in 0.11 seconds

For each user in our user::virtual class, we need to create:
We could declare separate resources to implement all of these for each user, but it's much easier to create a definition instead, which wraps them into a single resource. By creating a new module for our definition, we can refer to ssh_user from anywhere (in any scope):
define ssh_user ($key, $keytype) { 
  user { $name:
    ensure     => present,
  }
After we create the user, we can then create the home directory; we need the user first so that when we assign ownership, we can use the username, owner => $name:
  file { "/home/${name}":
    ensure => directory,
    mode => '0700',
    owner => $name,
    require => User["$name"]
  }
Tip
Puppet can create the users' home directory using the managehome attribute to the user resource. Relying on this mechanism is problematic in practice, as it does not account for users that were created outside of Puppet without home directories.
Next, we need to ensure that the .ssh directory exists within the home directory of the user. We require the home directory, File["/home/${name}"], since that needs to exist before we create this subdirectory. This implies that the user already exists because the home directory required the user:
  file { "/home/${name}/.ssh":
    ensure => directory,
    mode   => '0700',
    owner  => $name ,
    require => File["/home/${name}"],
  }
Finally, we create the ssh_authorized_key resource, again requiring the containing folder (File["/home/${name}/.ssh"]). We use the $key and $keytype variables to assign the key and type parameters to the ssh_authorized_key type as follows:
  ssh_authorized_key { "${name}_key":
    key     => $key,
    type    => "$keytype",
    user    => $name,
    require => File["/home/${name}/.ssh"],
  }
}
We passed the $key and $keytype variables when we defined the ssh_user resource for thomas:
@ssh_user { 'thomas':
  key => 'AAAAB3NzaC1yc2E...XaWM5sX0z',
  keytype => 'ssh-rsa'
}
Tip
The value for key, in the preceding code snippet, is the ssh key's public key value; it is usually stored in an id_rsa.pub file.
Now, with everything defined, we just need to call realize on thomas for all these resources to take effect:
realize(Ssh_user['thomas'])
Notice that this time the virtual resource we're realizing is not simply the user resource, as before, but the ssh_user defined type we created, which includes the user and the related resources needed to set up the SSH access:
Notice: /Stage[main]/User::Virtual/Ssh_user[thomas]/User[thomas]/ensure: created
Notice: /Stage[main]/User::Virtual/Ssh_user[thomas]/File[/home/thomas]/ensure: created
Notice: /Stage[main]/User::Virtual/Ssh_user[thomas]/File[/home/thomas/.ssh]/ensure: created
Notice: /Stage[main]/User::Virtual/Ssh_user[thomas]/Ssh_authorized_key[thomas_key]/ensure: created

Of course, you can add whatever resources you like to the ssh_user definition to have Puppet automatically create them for new users. We'll see an example of this in the next recipe, Managing users' customization files.


How it works...


For each user in our user::virtual class, we need to create:


The user account itself


The user's home directory and .ssh directory


The user's .ssh/authorized_keys file


We could declare separate resources to implement all of these for each user, but it's much easier to create a definition instead, which wraps them into a single resource. By creating a new module for our definition, we can refer to ssh_user from anywhere (in any scope):


define ssh_user ($key, $keytype) {
user { $name:
ensure => present,
}


After we create the user, we can then create the home directory; we need the user first so that when we assign ownership, we can use the username, owner => $name:


 file { "/home/${name}":
ensure => directory,
mode => '0700',
owner => $name,
require => User["$name"]
}

Tip
Puppet can create the users' home directory using the managehome attribute to the user resource. Relying on this mechanism is problematic in practice, as it does not account for users that were created outside of Puppet without home directories.
Next, we need to ensure that the .ssh directory exists within the home directory of the user. We require the home directory, File["/home/${name}"], since that needs to exist before we create this subdirectory. This implies that the user already exists because the home directory required the user:
  file { "/home/${name}/.ssh":
    ensure => directory,
    mode   => '0700',
    owner  => $name ,
    require => File["/home/${name}"],
  }
Finally, we create the ssh_authorized_key resource, again requiring the containing folder (File["/home/${name}/.ssh"]). We use the $key and $keytype variables to assign the key and type parameters to the ssh_authorized_key type as follows:
  ssh_authorized_key { "${name}_key":
    key     => $key,
    type    => "$keytype",
    user    => $name,
    require => File["/home/${name}/.ssh"],
  }
}
We passed the $key and $keytype variables when we defined the ssh_user resource for thomas:
@ssh_user { 'thomas':
  key => 'AAAAB3NzaC1yc2E...XaWM5sX0z',
  keytype => 'ssh-rsa'
}
Tip
The value for key, in the preceding code snippet, is the ssh key's public key value; it is usually stored in an id_rsa.pub file.
Now, with everything defined, we just need to call realize on thomas for all these resources to take effect:
realize(Ssh_user['thomas'])
Notice that this time the virtual resource we're realizing is not simply the user resource, as before, but the ssh_user defined type we created, which includes the user and the related resources needed to set up the SSH access:
Notice: /Stage[main]/User::Virtual/Ssh_user[thomas]/User[thomas]/ensure: created
Notice: /Stage[main]/User::Virtual/Ssh_user[thomas]/File[/home/thomas]/ensure: created
Notice: /Stage[main]/User::Virtual/Ssh_user[thomas]/File[/home/thomas/.ssh]/ensure: created
Notice: /Stage[main]/User::Virtual/Ssh_user[thomas]/Ssh_authorized_key[thomas_key]/ensure: created

Of course, you can add whatever resources you like to the ssh_user definition to have Puppet automatically create them for new users. We'll see an example of this in the next recipe, Managing users' customization files.


There's more...


Of course, you can add whatever resources you like to the ssh_user definition to have Puppet automatically create them for new users. We'll see an example of this in the next recipe, Managing users' customization files.





Users tend to customize their shell environments, terminal colors, aliases, and so forth. This is usually achieved by a number of dotfiles in their home directory, for example, .bash_profile or .vimrc.
You can use Puppet to synchronize and update each user's dotfiles across a number of machines by extending the virtual user setup we developed throughout this chapter. We'll start a new module, admin_user and use the file types, recurse attribute to copy files into each user's home directory.
Here's what you need to do:
  1. Create the admin_user defined type (define admin_user) in the modules/admin_user/manifests/init.pp file as follows:
    define admin_user ($key, $keytype, $dotfiles = false) { 
  $username = $name
  user { $username:
    ensure     => present,
  }
  file { "/home/${username}/.ssh":
    ensure  => directory,
    mode    => '0700',
    owner   => $username,
    group   => $username,
    require => File["/home/${username}"],
  }
  ssh_authorized_key { "${username}_key":
    key     => $key,
    type    => "$keytype",
    user    => $username,
    require => File["/home/${username}/.ssh"],
  }
  # dotfiles
  if $dotfiles == false {
    # just create the directory
    file { "/home/${username}":
      ensure  => 'directory',
      mode    => '0700',
      owner   => $username,
      group   => $username,
      require => User["$username"]
    }
  } else {
    # copy in all the files in the subdirectory
    file { "/home/${username}":
      recurse => true,
      mode    => '0700',
      owner   => $username,
      group   => $username,
      source  => "puppet:///modules/admin_user/${username}",
      require => User["$username"]
    }
  }
}
  2. Modify the file modules/user/manifests/sysadmins.pp as follows:
    class user::sysadmins {
  realize(Admin_user['thomas'])
}
  3. Alter the definition of thomas in modules/user/manifests/virtual.pp as follows:
    @ssh_user { 'thomas':
  key => 'AAAAB3NzaC1yc2E...XaWM5sX0z',
  keytype => 'ssh-rsa',
  dotfiles => true
}
  4. Create a subdirectory in the admin_user module for the file of user thomas:
    $ mkdir -p modules/admin_user/files/thomas

  5. Create dotfiles for the user thomas in the directory you just created:
    $ echo "alias vi=vim" > modules/admin_user/files/thomas/.bashrc
$ echo "set tabstop=2" > modules/admin_user/files/thomas/.vimrc

  6. Make sure your site.pp file reads as follows:
    node 'cookbook' {
  include user::virtual
  include user::sysadmins
}
  7. Run Puppet:
    cookbook# puppet agent -t
Info: Caching catalog for cookbook.example.com
Info: Applying configuration version '1413266235'
Notice: /Stage[main]/User::Virtual/Admin_user[thomas]/User[thomas]/ensure: created
Notice: /Stage[main]/User::Virtual/Admin_user[thomas]/File[/home/thomas]/ensure: created
Notice: /Stage[main]/User::Virtual/Admin_user[thomas]/File[/home/thomas/.vimrc]/ensure: defined content as '{md5}cb2af2d35b18b5ac2539057bd429d3ae'
Notice: /Stage[main]/User::Virtual/Admin_user[thomas]/File[/home/thomas/.bashrc]/ensure: defined content as '{md5}033c3484e4b276e0641becc3aa268a3a'
Notice: /Stage[main]/User::Virtual/Admin_user[thomas]/File[/home/thomas/.ssh]/ensure: created
Notice: /Stage[main]/User::Virtual/Admin_user[thomas]/Ssh_authorized_key[thomas_key]/ensure: created
Notice: Finished catalog run in 0.36 seconds

We created a new admin_user definition, which defines the home directory recursively if $dotfiles is not false (the default value):
  if $dotfiles == 'false' {
    # just create the directory
    file { "/home/${username}":
      ensure  => 'directory',
      mode    => '0700',
      owner   => $username,
      group   => $username,
      require => User["$username"]
    }
  } else {
    # copy in all the files in the subdirectory
    file { "/home/${username}":
      recurse => true,
      mode    => '0700',
      owner   => $username,
      group   => $username,
      source  => "puppet:///modules/admin_user/${username}",
      require => User["$username"]
    }
  }
We created a directory to hold the user's dotfiles within the admin_user module; all the files within that directory will be copied into the user's home directory, as shown in the puppet run output in the following command line:
Notice: /Stage[main]/User::Virtual/Admin_user[thomas]/File[/home/thomas/.vimrc]/ensure: defined content as '{md5}cb2af2d35b18b5ac2539057bd429d3ae'
Notice: /Stage[main]/User::Virtual/Admin_user[thomas]/File[/home/thomas/.bashrc]/ensure: defined content as '{md5}033c3484e4b276e0641becc3aa268a3a'

Using the recurse option allows us to add as many dotfiles as we wish for each user without having to modify the definition of the user.
We could specify that the source attribute of the home directory is a directory where users can place their own dotfiles. This way, each user could modify their own dotfiles and have them transferred to all the nodes in the network without our involvement.


How to do it...


Here's what you 

need to do:
  1. Create the admin_user defined type (define admin_user) in the modules/admin_user/manifests/init.pp file as follows:
    define admin_user ($key, $keytype, $dotfiles = false) { 
  $username = $name
  user { $username:
    ensure     => present,
  }
  file { "/home/${username}/.ssh":
    ensure  => directory,
    mode    => '0700',
    owner   => $username,
    group   => $username,
    require => File["/home/${username}"],
  }
  ssh_authorized_key { "${username}_key":
    key     => $key,
    type    => "$keytype",
    user    => $username,
    require => File["/home/${username}/.ssh"],
  }
  # dotfiles
  if $dotfiles == false {
    # just create the directory
    file { "/home/${username}":
      ensure  => 'directory',
      mode    => '0700',
      owner   => $username,
      group   => $username,
      require => User["$username"]
    }
  } else {
    # copy in all the files in the subdirectory
    file { "/home/${username}":
      recurse => true,
      mode    => '0700',
      owner   => $username,
      group   => $username,
      source  => "puppet:///modules/admin_user/${username}",
      require => User["$username"]
    }
  }
}
  2. Modify the file modules/user/manifests/sysadmins.pp as follows:
    class user::sysadmins {
  realize(Admin_user['thomas'])
}
  3. Alter the definition of thomas in modules/user/manifests/virtual.pp as follows:
    @ssh_user { 'thomas':
  key => 'AAAAB3NzaC1yc2E...XaWM5sX0z',
  keytype => 'ssh-rsa',
  dotfiles => true
}
  4. Create a subdirectory in the admin_user module for the file of user thomas:
    $ mkdir -p modules/admin_user/files/thomas

  5. Create dotfiles for the user thomas in the directory you just created:
    $ echo "alias vi=vim" > modules/admin_user/files/thomas/.bashrc
$ echo "set tabstop=2" > modules/admin_user/files/thomas/.vimrc

  6. Make sure your site.pp file reads as follows:
    node 'cookbook' {
  include user::virtual
  include user::sysadmins
}
  7. Run Puppet:
    cookbook# puppet agent -t
Info: Caching catalog for cookbook.example.com
Info: Applying configuration version '1413266235'
Notice: /Stage[main]/User::Virtual/Admin_user[thomas]/User[thomas]/ensure: created
Notice: /Stage[main]/User::Virtual/Admin_user[thomas]/File[/home/thomas]/ensure: created
Notice: /Stage[main]/User::Virtual/Admin_user[thomas]/File[/home/thomas/.vimrc]/ensure: defined content as '{md5}cb2af2d35b18b5ac2539057bd429d3ae'
Notice: /Stage[main]/User::Virtual/Admin_user[thomas]/File[/home/thomas/.bashrc]/ensure: defined content as '{md5}033c3484e4b276e0641becc3aa268a3a'
Notice: /Stage[main]/User::Virtual/Admin_user[thomas]/File[/home/thomas/.ssh]/ensure: created
Notice: /Stage[main]/User::Virtual/Admin_user[thomas]/Ssh_authorized_key[thomas_key]/ensure: created
Notice: Finished catalog run in 0.36 seconds

We created a new admin_user definition, which defines the home directory recursively if $dotfiles is not false (the default value):
  if $dotfiles == 'false' {
    # just create the directory
    file { "/home/${username}":
      ensure  => 'directory',
      mode    => '0700',
      owner   => $username,
      group   => $username,
      require => User["$username"]
    }
  } else {
    # copy in all the files in the subdirectory
    file { "/home/${username}":
      recurse => true,
      mode    => '0700',
      owner   => $username,
      group   => $username,
      source  => "puppet:///modules/admin_user/${username}",
      require => User["$username"]
    }
  }
We created a directory to hold the user's dotfiles within the admin_user module; all the files within that directory will be copied into the user's home directory, as shown in the puppet run output in the following command line:
Notice: /Stage[main]/User::Virtual/Admin_user[thomas]/File[/home/thomas/.vimrc]/ensure: defined content as '{md5}cb2af2d35b18b5ac2539057bd429d3ae'
Notice: /Stage[main]/User::Virtual/Admin_user[thomas]/File[/home/thomas/.bashrc]/ensure: defined content as '{md5}033c3484e4b276e0641becc3aa268a3a'

Using the recurse option allows us to add as many dotfiles as we wish for each user without having to modify the definition of the user.
We could specify that the source attribute of the home directory is a directory where users can place their own dotfiles. This way, each user could modify their own dotfiles and have them transferred to all the nodes in the network without our involvement.


How it works...


We created a

 new admin_user definition, which defines the home directory recursively if $dotfiles is not false (the default value):
  if $dotfiles == 'false' {
    # just create the directory
    file { "/home/${username}":
      ensure  => 'directory',
      mode    => '0700',
      owner   => $username,
      group   => $username,
      require => User["$username"]
    }
  } else {
    # copy in all the files in the subdirectory
    file { "/home/${username}":
      recurse => true,
      mode    => '0700',
      owner   => $username,
      group   => $username,
      source  => "puppet:///modules/admin_user/${username}",
      require => User["$username"]
    }
  }
We created a directory to hold the user's dotfiles within the admin_user module; all the files within that directory will be copied into the user's home directory, as shown in the puppet run output in the following command line:
Notice: /Stage[main]/User::Virtual/Admin_user[thomas]/File[/home/thomas/.vimrc]/ensure: defined content as '{md5}cb2af2d35b18b5ac2539057bd429d3ae'
Notice: /Stage[main]/User::Virtual/Admin_user[thomas]/File[/home/thomas/.bashrc]/ensure: defined content as '{md5}033c3484e4b276e0641becc3aa268a3a'

Using the recurse option allows us to add as many dotfiles as we wish for each user without having to modify the definition of the user.
We could specify that the source attribute of the home directory is a directory where users can place their own dotfiles. This way, each user could modify their own dotfiles and have them transferred to all the nodes in the network without our involvement.


There's more...


We could specify that the source attribute of the home directory is a directory where users can place their own dotfiles. This way, each user could modify their own dotfiles and have them transferred to all the nodes in the network without our involvement.



See also


The Managing users with virtual resources recipe in this chapter





All our recipes up to this point have dealt with a single machine. It is possible with Puppet to have resources from one node affect another node. This interaction is managed with exported resources. Exported resources are just like any resource you might define for a node but instead of applying to the node on which they were created, they are exported for use by all nodes in the environment. Exported resources can be thought of as virtual resources that go one step further and exist beyond the node on which they were defined.
There are two actions with exported resources. When an exported resource is created, it is said to be defined. When all the exported resources are harvested, they are said to be collected. Defining exported resources is similar to virtual resources; the resource in question has two @ symbols prepended. For example, to define a file resource as external, use @@file. Collecting resources is done with the space ship operator, <<| |>>; this is thought to look like a spaceship. To collect the exported file resource (@@file), you would use File <<| |>>.
There are many examples that use exported resources; the most common one involves SSH host keys. Using exported resources, it is possible to have every machine that is running Puppet share their SSH host keys with the other connected nodes. The idea here is that each machine exports its own host key and then collects all the keys from the other machines. In our example, we will create two classes; first, a class that exports the SSH host key from every node. We will include this class in our base class. The second class will be a collector class, which collects the SSH host keys. We will apply this class to our Jumpboxes or SSH login servers.
Note
Jumpboxes are machines that have special firewall rules to allow them to log in to different locations.
To use exported resources, you will need to enable storeconfigs on your Puppet masters. It is possible to use exported resources with a masterless (decentralized) deployment; however, we will assume you are using a centralized model for this example. In Chapter 2, Puppet Infrastructure, we configured puppetdb using the puppetdb module from the forge. It is possible to use other backends if you desire; however, all of these except puppetdb are deprecated. More information is available at the following link: http://projects.puppetlabs.com/projects/puppet/wiki/Using_Stored_Configuration.
Ensure your Puppet masters are configured to use puppetdb as a storeconfigs container.
We'll create an ssh_host class to export the ssh keys of a host and ensure that it is included in our base class.
  1. Create the first class, base::ssh_host, which we will include in our base class:
    class base::ssh_host {
  @@sshkey{"$::fqdn":
    ensure       => 'present',
    host_aliases => ["$::hostname","$::ipaddress"],
    key          => $::sshdsakey,
    type         => 'dsa',
  }
}
  2. Remember to include this class from inside the base class definition:
    class base {
  ...
  include ssh_host
}
  3. Create a definition for jumpbox, either in a class or within the node definition for jumpbox:
    node 'jumpbox' {
  Sshkey <<| |>>
}
  4. Now run Puppet on a few nodes to create the exported resources. In my case, I ran Puppet on my Puppet server and my second example node (node2). Finally, run Puppet on jumpbox to verify that the SSH host keys for our other nodes are collected:
    [root@jumpbox ~]# puppet agent -t 
Info: Caching catalog for jumpbox.example.com
Info: Applying configuration version '1413176635'
Notice: /Stage[main]/Main/Node[jumpbox]/Sshkey[node2.example.com]/ensure: created
Notice: /Stage[main]/Main/Node[jumpbox]/Sshkey[puppet]/ensure: created
Notice: Finished catalog run in 0.08 seconds

We created an sshkey resource for the node using the facter facts fqdn, hostname, ipaddress, and sshdsakey. We use the fqdn as the title for our exported resource because each exported resource must have a unique name. We can assume the fqdn of a node will be unique within our organization (although sometimes they may not be; Puppet can be good at finding out such things when you least expect it). We then go on to define aliases by which our node may be known. We use the hostname variable for one alias and the main IP address of the machine as the other. If you had other naming conventions for your nodes, you could include other aliases here. We assume that hosts are using DSA keys, so we use the sshdsakey variable in our definition. In a large installation, you would wrap this definition in tests to ensure the DSA keys existed. You would also use the RSA keys if they existed as well.
With the sshkey resource defined and exported, we then created a jumpbox node definition. In this definition, we used the spaceship syntax Sshkey <<| |>> to collect all defined exported sshkey resources.
When defining the exported resources, you can add tag attributes to the resource to create subsets of exported resources. For example, if you had a development and production area of your network, you could create different groups of sshkey resources for each area as shown in the following code snippet:
@@sshkey{"$::fqdn":
    host_aliases => ["$::hostname","$::ipaddress"],
    key          => $::sshdsakey,
    type         => 'dsa',
    tag          => "$::environment",
  }
You could then modify jumpbox to only collect resources for production, for example, as follows:
Sshkey <<| tag == 'production' |>>
Two important things to remember when working with exported resources: first, every resource must have a unique name across your installation. Using the fqdn domain name within the title is usually enough to keep your definitions unique. Second, any resource can be made virtual. Even defined types that you created may be exported. Exported resources can be used to achieve some fairly complex configurations that automatically adjust when machines change.
Note
One word of caution when working with an extremely large number of nodes (more than 5,000) is that exported resources can take a long time to collect and apply, particularly if each exported resource creates a file.


Getting ready


To use exported

 resources, you will need to enable storeconfigs on your Puppet masters. It is possible to use exported resources with a masterless (decentralized) deployment; however, we will assume you are using a centralized model for this example. In Chapter 2, Puppet Infrastructure, we configured puppetdb using the puppetdb module from the forge. It is possible to use other backends if you desire; however, all of these except puppetdb are deprecated. More information is available at the following link: http://projects.puppetlabs.com/projects/puppet/wiki/Using_Stored_Configuration.
Ensure your Puppet masters are configured to use puppetdb as a storeconfigs container.
We'll create an ssh_host class to export the ssh keys of a host and ensure that it is included in our base class.
  1. Create the first class, base::ssh_host, which we will include in our base class:
    class base::ssh_host {
  @@sshkey{"$::fqdn":
    ensure       => 'present',
    host_aliases => ["$::hostname","$::ipaddress"],
    key          => $::sshdsakey,
    type         => 'dsa',
  }
}
  2. Remember to include this class from inside the base class definition:
    class base {
  ...
  include ssh_host
}
  3. Create a definition for jumpbox, either in a class or within the node definition for jumpbox:
    node 'jumpbox' {
  Sshkey <<| |>>
}
  4. Now run Puppet on a few nodes to create the exported resources. In my case, I ran Puppet on my Puppet server and my second example node (node2). Finally, run Puppet on jumpbox to verify that the SSH host keys for our other nodes are collected:
    [root@jumpbox ~]# puppet agent -t 
Info: Caching catalog for jumpbox.example.com
Info: Applying configuration version '1413176635'
Notice: /Stage[main]/Main/Node[jumpbox]/Sshkey[node2.example.com]/ensure: created
Notice: /Stage[main]/Main/Node[jumpbox]/Sshkey[puppet]/ensure: created
Notice: Finished catalog run in 0.08 seconds

We created an sshkey resource for the node using the facter facts fqdn, hostname, ipaddress, and sshdsakey. We use the fqdn as the title for our exported resource because each exported resource must have a unique name. We can assume the fqdn of a node will be unique within our organization (although sometimes they may not be; Puppet can be good at finding out such things when you least expect it). We then go on to define aliases by which our node may be known. We use the hostname variable for one alias and the main IP address of the machine as the other. If you had other naming conventions for your nodes, you could include other aliases here. We assume that hosts are using DSA keys, so we use the sshdsakey variable in our definition. In a large installation, you would wrap this definition in tests to ensure the DSA keys existed. You would also use the RSA keys if they existed as well.
With the sshkey resource defined and exported, we then created a jumpbox node definition. In this definition, we used the spaceship syntax Sshkey <<| |>> to collect all defined exported sshkey resources.
When defining the exported resources, you can add tag attributes to the resource to create subsets of exported resources. For example, if you had a development and production area of your network, you could create different groups of sshkey resources for each area as shown in the following code snippet:
@@sshkey{"$::fqdn":
    host_aliases => ["$::hostname","$::ipaddress"],
    key          => $::sshdsakey,
    type         => 'dsa',
    tag          => "$::environment",
  }
You could then modify jumpbox to only collect resources for production, for example, as follows:
Sshkey <<| tag == 'production' |>>
Two important things to remember when working with exported resources: first, every resource must have a unique name across your installation. Using the fqdn domain name within the title is usually enough to keep your definitions unique. Second, any resource can be made virtual. Even defined types that you created may be exported. Exported resources can be used to achieve some fairly complex configurations that automatically adjust when machines change.
Note
One word of caution when working with an extremely large number of nodes (more than 5,000) is that exported resources can take a long time to collect and apply, particularly if each exported resource creates a file.


How to do it...


We'll create an ssh_host class to export the ssh keys of a host and ensure that it is included in our base class.


Create the first class, base::ssh_host, which we will include in our base class:
class base::ssh_host {
  @@sshkey{"$::fqdn":
    ensure       => 'present',
    host_aliases => ["$::hostname","$::ipaddress"],
    key          => $::sshdsakey,
    type         => 'dsa',
  }
}



Remember to include this class from inside the base class definition:
class base {
  ...
  include ssh_host
}



Create a definition for jumpbox, either in a class or within the node definition for jumpbox:
node 'jumpbox' {
  Sshkey <<| |>>
}



Now run Puppet on a few nodes to create the exported resources. In my case, I ran Puppet on my Puppet server and my second example node (node2). Finally, run Puppet on jumpbox to verify that the SSH host keys for our other nodes are collected:
[root@jumpbox ~]# puppet agent -t 
Info: Caching catalog for jumpbox.example.com
Info: Applying configuration version '1413176635'
Notice: /Stage[main]/Main/Node[jumpbox]/Sshkey[node2.example.com]/ensure: created
Notice: /Stage[main]/Main/Node[jumpbox]/Sshkey[puppet]/ensure: created
Notice: Finished catalog run in 0.08 seconds



We created an sshkey resource for the node using the facter facts fqdn, hostname, ipaddress, and sshdsakey. We use the fqdn as the title for our exported resource because each exported resource must have a unique name. We can assume the fqdn of a node will be unique within our organization (although sometimes they may not be; Puppet can be good at finding out such things when you least expect it). We then go on to define aliases by which our node may be known. We use the hostname variable for one alias and the main IP address of the machine as the other. If you had other naming conventions for your nodes, you could include other aliases here. We assume that hosts are using DSA keys, so we use the sshdsakey variable in our definition. In a large installation, you would wrap this definition in tests to ensure the DSA keys existed. You would also use the RSA keys if they existed as well.
With the sshkey resource defined and exported, we then created a jumpbox node definition. In this definition, we used the spaceship syntax Sshkey <<| |>> to collect all defined exported sshkey resources.
When defining the exported resources, you can add tag attributes to the resource to create subsets of exported resources. For example, if you had a development and production area of your network, you could create different groups of sshkey resources for each area as shown in the following code snippet:
@@sshkey{"$::fqdn":
    host_aliases => ["$::hostname","$::ipaddress"],
    key          => $::sshdsakey,
    type         => 'dsa',
    tag          => "$::environment",
  }
You could then modify jumpbox to only collect resources for production, for example, as follows:
Sshkey <<| tag == 'production' |>>
Two important things to remember when working with exported resources: first, every resource must have a unique name across your installation. Using the fqdn domain name within the title is usually enough to keep your definitions unique. Second, any resource can be made virtual. Even defined types that you created may be exported. Exported resources can be used to achieve some fairly complex configurations that automatically adjust when machines change.
Note
One word of caution when working with an extremely large number of nodes (more than 5,000) is that exported resources can take a long time to collect and apply, particularly if each exported resource creates a file.


How it works...


We created an sshkey resource for 

the node using the facter facts fqdn, hostname, ipaddress, and sshdsakey. We use the fqdn as the title for our exported resource because each exported resource must have a unique name. We can assume the fqdn of a node will be unique within our organization (although sometimes they may not be; Puppet can be good at finding out such things when you least expect it). We then go on to define aliases by which our node may be known. We use the hostname variable for one alias and the main IP address of the machine as the other. If you had other naming conventions for your nodes, you could include other aliases here. We assume that hosts are using DSA keys, so we use the sshdsakey variable in our definition. In a large installation, you would wrap this definition in tests to ensure the DSA keys existed. You would also use the RSA keys if they existed as well.
With the sshkey resource defined and exported, we then created a jumpbox node definition. In this definition, we used the spaceship syntax Sshkey <<| |>> to collect all defined exported sshkey resources.
When defining the exported resources, you can add tag attributes to the resource to create subsets of exported resources. For example, if you had a development and production area of your network, you could create different groups of sshkey resources for each area as shown in the following code snippet:
@@sshkey{"$::fqdn":
    host_aliases => ["$::hostname","$::ipaddress"],
    key          => $::sshdsakey,
    type         => 'dsa',
    tag          => "$::environment",
  }
You could then modify jumpbox to only collect resources for production, for example, as follows:
Sshkey <<| tag == 'production' |>>
Two important things to remember when working with exported resources: first, every resource must have a unique name across your installation. Using the fqdn domain name within the title is usually enough to keep your definitions unique. Second, any resource can be made virtual. Even defined types that you created may be exported. Exported resources can be used to achieve some fairly complex configurations that automatically adjust when machines change.
Note
One word of caution when working with an extremely large number of nodes (more than 5,000) is that exported resources can take a long time to collect and apply, particularly if each exported resource creates a file.


There's more...


When defining the exported resources, you can add tag attributes to the resource to create subsets of exported resources. For example, if you had a development and production area of your network, you could create different groups of sshkey resources for each area as shown in the following code snippet:


@@sshkey{"$::fqdn":
host_aliases => ["$::hostname","$::ipaddress"],
key => $::sshdsakey,
type => 'dsa',
tag => "$::environment",
}


You could then modify jumpbox to only collect resources for production, for example, as follows:


Sshkey <<| tag == 'production' |>>


Two important things to remember when working with exported resources: first, every resource must have a unique name across your installation. Using the fqdn domain name within the title is usually enough to keep your definitions unique. Second, any resource can be made virtual. Even defined types that you created may be exported. Exported resources can be used to 

achieve some fairly complex configurations that automatically adjust when machines change.
Note
One word of caution when working with an extremely large number of nodes (more than 5,000) is that exported resources can take a long time to collect and apply, particularly if each exported resource creates a file.




















Previous Section





End of Section 6





Next
Section