Book Image

Mastering Chef

By : Mayank Joshi
Book Image

Mastering Chef

By: Mayank Joshi

Overview of this book

Table of Contents (20 chapters)
Mastering Chef
About the Author
About the Reviewers
Knife and Its Associated Plugins
Data Bags and Templates

Setting up a work environment

As we saw earlier, the Chef ecosystem comprises of three components: chef-server, chef-client, and a developer workstation.

We'll be developing all our beautiful Chef codes on our workstation. As we are developing a code, it's good practice to keep our code in some version control system such as git/svn/mercurial and so on. We'll choose Git for our purpose and I'll presume you've a repository called chef-repo that is being tracked by Git.

The following software should be installed on your machine before you try to set up your workstation:

  • Ruby (Preferably, 1.9.x).

  • We need Chef and Knife installed on our workstation and it's pretty easy to go about installing Chef along with Knife using the Ruby gems. Just open up a terminal and issue the command:

    #gem install chef
  • Once Chef is installed, create a .chef folder in your home directory and create a knife.rb file in it.

Knife is a tool using which we'll use to communicate with a chef-server. Knife can be used for lots of purposes such as managing cookbooks, nodes, API clients, roles, environments, and so on. Knife also comes with plugins that allow it to be used for various other useful purposes. We'll learn more about them in later chapters.

Knife needs the knife.rb file present in the $HOME/.chef folder. The following is a sample knife.rb file:

log_level               :info
log_location            STDOUT
node_name               'NAME_OF_YOUR_CHOICE'
client_key              '~/.chef/NAME_OF_YOUR_CHOICE.pem'
validation_client_name  'chef-validator'
validation_key          '~/.chef/validation.pem'
chef_server_url         ''
cache_type              'BasicFile'
cache_options           (:path => '~/.chef/checksums')
cookbook_path           [ '~/code/chef-repo/cookbooks' ]

Connect to your chef-server web interface and visit the client section and create a new client with a name of your choice (ensure that no client with the same name exists on the chef-server):

Once you've created the client, a chef-server will respond with a public/private key pair as shown in the following screenshot:

Copy the contents of the private key and store them in ~/.chef/<NAME_OF_YOUR_CHOICE>.pem

Also, copy the private key for the chef-validator (/etc/chef/validation.pem) from the chef-server to ~/.chef/validation.pem.

Specify NAME_OF_YOUR_CHOICE as the node name.

As you can see, we've specified cookbook_path to be ~/code/chef-repo/cookbooks. I'm presuming that you'll be storing your Chef cookbooks inside this folder.

Create the following directory structure inside ~/code/chef-repo:

    ├── cookbooks
    ├── data_bags
    ├── environments
    └── roles

The cookbooks directory will hold our cookbooks, the data_bags directory will contain data bags, the environments directory will contain configuration files for different environments, and the roles directory will contain files associated with different roles.

Once you've created these directories, commit them to your Git repository.

Now, let's try to see if we are able to make use of the Knife executable and query the Chef server:

$knife client list

This command will list all the available API clients registered with the chef-server. As you can see, chef-eg01 is a newly created client and it's now registered with the chef-server.

Knife caches the checksum of Ruby and ERB files when performing a cookbook syntax check with knife cookbook test or knife cookbook upload. The cache_type variable defines which type of cache to make use of. The most used type is BasicFile and it's probably best to leave it at that.

The cache_options is a hash for options related to caching. For BasicFile, :path should be the location on the filesystem where Knife has write access.

If you want the Knife cookbook to create a command to prefill values for copyright and e-mail in comments, you can also specify the following options in your knife.rb file:

cookbook_copyright "Company name"
cookbook_email "Email address"

With this setup, now we are ready to start creating new cookbooks, roles, and environments, and manage them along with nodes and clients using Knife from our workstation.

Before we jump into cookbook creation and other exciting stuff, we need to ensure that we follow a test-driven approach to our Chef development. We will make use of test-kitchen to help us write Chef cookbooks that are tested thoroughly before being pushed to a chef-server.

test-kitchen can be installed as a gem:

$ gem install test-kitchen

Also, download Vagrant from and install it.

If you want some help, use the help option of the kitchen command:

$ kitchen help
  kitchen console                         # Kitchen Console!
  kitchen converge [INSTANCE|REGEXP|all]  # Converge one or more instances
  kitchen create [INSTANCE|REGEXP|all]    # Create one or more instances
  kitchen destroy [INSTANCE|REGEXP|all]   # Destroy one or more instances
  kitchen diagnose [INSTANCE|REGEXP|all]  # Show computed diagnostic configuration
  kitchen driver                          # Driver subcommands
  kitchen driver create [NAME]            # Create a new Kitchen Driver gem project
  kitchen driver discover                 # Discover Test Kitchen drivers published on RubyGems
  kitchen driver help [COMMAND]           # Describe subcommands or one specific subcommand
  kitchen help [COMMAND]                  # Describe available commands or one specific command
  kitchen init                            # Adds some configuration to your cookbook so Kitchen can rock
  kitchen list [INSTANCE|REGEXP|all]      # Lists one or more instances
  kitchen login INSTANCE|REGEXP           # Log in to one instance
  kitchen setup [INSTANCE|REGEXP|all]     # Setup one or more instances
  kitchen test [INSTANCE|REGEXP|all]      # Test one or more instances
  kitchen verify [INSTANCE|REGEXP|all]    # Verify one or more instances
  kitchen version                         # Print Kitchen's version information

Now, let's create a new cookbook called passenger-nginx:

$knife cookbook create passenger-nginx

Now, we'll add test-kitchen to our project using the init subcommand:

$ kitchen init
create .kitchen.yml
create test/integration/default
run gem install kitchen-vagrant from "."
Fetching: kitchen-vagrant-0.14.0.gem (100%)
Successfully installed kitchen-vagrant-0.14.0
Parsing documentation for kitchen-vagrant-0.14.0
Installing ri documentation for kitchen-vagrant-0.14.0
Done installing documentation for kitchen-vagrant after 0 seconds
1 gem installed

The kitchen init command has created a configuration file called .kitchen.yml, along with a test/integration/default directory.

It also went on to install a gem called kitchen-vagrant. kitchen needs a virtual machine to test run the chef code, and drivers are responsible for managing virtual machines. By default, kitchen makes use of Vagrant to manage the virtual machine.

Let's see what we have in our configuration file, kitchen.yml:

$ cat .kitchen.yml
  name: vagrant
  name: chef_solo
  - name: ubuntu-12.04
  - name: centos-6.4
  - name: default
      - recipe[cb-test1::default]

The file is divided into four sections:

  • Driver: This is where we set up basic stuff such as the SSH username and credentials. Under this section, we've a name property with a vagrant value. This tells kitchen to make use of the kitchen-vagrant driver.

  • Provisioner: This tells kitchen to make use of a chef-solo to apply the cookbook to a newly created virtual machine.

  • Platforms: This lists the operating systems on which we want to run our code.

  • Suites: Here we describe what we wish to test.

Now, let's see what we have on our hands:

$ kitchen list
Instance             Driver   Provisioner  Last Action
default-ubuntu-1204  Vagrant  ChefSolo     <Not Created>
default-centos-64    Vagrant  ChefSolo     <Not Created>

As you can see, it's listing two instances: default-ubuntu-1204 and default-centos-64. These names are a combination of the suite name and the platform name.

Now, let's spin up one instance to see what happens:

$ kitchen create default-ubuntu-1204
-----> Starting Kitchen (v1.2.1)
-----> Creating <default-ubuntu-1204>...
       Bringing machine 'default' up with 'virtualbox' provider...
       ==> default: Box 'opscode-ubuntu-12.04' could not be found. Attempting to find and install...
           default: Box Provider: virtualbox
           default: Box Version: >= 0
       ==> default: Adding box 'opscode-ubuntu-12.04' (v0) for provider: virtualbox
           default: Downloading:
       ==> default: Successfully added box 'opscode-ubuntu-12.04' (v0) for 'virtualbox'!
       ==> default: Importing base box 'opscode-ubuntu-12.04'...
       ==> default: Matching MAC address for NAT networking...
       ==> default: Setting the name of the VM: default-ubuntu-1204_default_1398006642518_53572
       ==> default: Clearing any previously set network interfaces...
       ==> default: Preparing network interfaces based on configuration...
           default: Adapter 1: nat
       ==> default: Forwarding ports...
           default: 22 => 2222 (adapter 1)
       ==> default: Running 'pre-boot' VM customizations...
       ==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...           default: SSH address:
           default: SSH username: vagrant
           default: SSH auth method: private key
           default: Warning: Connection timeout. Retrying...
       ==> default: Machine booted and ready!
       ==> default: Checking for guest additions in VM...
       ==> default: Setting hostname...
       Vagrant instance <default-ubuntu-1204> created.
       Finished creating <default-ubuntu-1204> (4m4.17s).
-----> Kitchen is finished. (4m4.71s)

So, this leads to the downloading of a virtual machine image for Ubuntu 12.04 and, eventually, the machine boots up. The default username for SSH connection is vagrant.

Let us check the status of our instance again:

$ kitchen list
Instance             Driver   Provisioner  Last Action
default-ubuntu-1204  Vagrant  ChefSolo     Created
default-centos-64    Vagrant  ChefSolo     <Not Created>

So, our Ubuntu instance is up and running. Now, let's add some meat to our recipe:

# Cookbook Name:: cb-test1
# Recipe:: default
# Copyright 2014, Sychonet
# All rights reserved - Do Not Redistribute

package "nginx"

log "Cool. So we have nginx installed"

So, now we've got our recipe ready, let's let test-kitchen run it in our instance now:

$ kitchen converge default-ubuntu-1204
-----> Starting Kitchen (v1.2.1)
-----> Converging <default-ubuntu-1204>...
       Preparing files for transfer
       Preparing current project directory as a cookbook
       Removing non-cookbook files before transfer
-----> Installing Chef Omnibus (true)
         to file /tmp/
       trying wget...
Downloading Chef  for ubuntu...      
  to file /tmp/      
trying wget...      
md5  cedd8a2df60a706e51f58adf8441971b      
sha256  af53e7ef602be6228dcbf68298e2613d3f37eb061975992abc6cd2d318e4a0c0      
downloaded metadata file looks valid...      
  to file /tmp/      
trying wget...      
Comparing checksum with sha256sum...      
Installing Chef       
installing with dpkg...      
Selecting previously unselected package chef.      
(Reading database ... 56035 files and directories currently installed.)      
Unpacking chef (from .../chef_11.12.2-1_amd64.deb) ...      
Setting up chef (11.12.2-1) ...      
Thank you for installing Chef!      
       Transfering files to <default-ubuntu-1204>
[2014-04-20T15:50:31+00:00] INFO: Forking chef instance to converge...      
[2014-04-20T15:50:31+00:00] WARN:       
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *       
SSL validation of HTTPS requests is disabled. HTTPS connections are still      
encrypted, but chef is not able to detect forged replies or man in the middle      
To fix this issue add an entry like this to your configuration file:      
  # Verify all HTTPS connections (recommended)      
  ssl_verify_mode :verify_peer      
  # OR, Verify only connections to chef-server      
  verify_api_cert true      
       To check your SSL configuration, or troubleshoot errors, you can use the
       `knife ssl check` command like so:
         knife ssl check -c /tmp/kitchen/solo.rb
       * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
Starting Chef Client, version 11.12.2      
[2014-04-20T15:50:31+00:00] INFO: *** Chef 11.12.2 ***      
[2014-04-20T15:50:31+00:00] INFO: Chef-client pid: 1225      
[2014-04-20T15:50:39+00:00] INFO: Setting the run_list to ["recipe[cb-test1::default]"] from CLI options      
[2014-04-20T15:50:39+00:00] INFO: Run List is [recipe[cb-test1::default]]      
[2014-04-20T15:50:39+00:00] INFO: Run List expands to [cb-test1::default]      
[2014-04-20T15:50:39+00:00] INFO: Starting Chef Run for default-ubuntu-1204      
[2014-04-20T15:50:39+00:00] INFO: Running start handlers      
[2014-04-20T15:50:39+00:00] INFO: Start handlers complete.      
Compiling Cookbooks...      
Converging 2 resources      
Recipe: cb-test1::default      
  * package[nginx] action install[2014-04-20T15:50:39+00:00] INFO: Processing package[nginx] action install (cb-test1::default line 10)      
           - install version 1.1.19-1ubuntu0.6 of package nginx
  * log[Cool. So we have nginx installed] action write[2014-04-20T15:50:52+00:00] INFO: Processing log[Cool. So we have nginx installed] action write (cb-test1::default line 12)      
[2014-04-20T15:50:52+00:00] INFO: Cool. So we have nginx installed      
[2014-04-20T15:50:52+00:00] INFO: Chef Run complete in 12.923797655 seconds      
Running handlers:      
[2014-04-20T15:50:52+00:00] INFO: Running report handlers      
Running handlers complete      
[2014-04-20T15:50:52+00:00] INFO: Report handlers complete      
Chef Client finished, 2/2 resources updated in 21.14983058 seconds      
       Finished converging <default-ubuntu-1204> (2m10.10s).
-----> Kitchen is finished. (2m10.41s)

So, here is what happened under the hood when kitchen converge was executed:

  • Chef was installed on an Ubuntu instance

  • Our cb-test1 cookbook and a chef-solo configuration were uploaded to an Ubuntu instance.

  • The Chef run was initiated using run_list and attributes defined in .kitchen.yml

If the exit code of the kitchen command is 0, then the command run was successful. If it's not 0, then any part of the operation associated with the command was not successful.

Let's check the status of our instance once more:

$ kitchen list
Instance             Driver   Provisioner  Last Action
default-ubuntu-1204  Vagrant  ChefSolo     Converged
default-centos-64    Vagrant  ChefSolo     <Not Created>

So, our instance is converged, but we still don't know if nginx was installed successfully or not. One way to check this is to log in to the instance using the following command:

$ kitchen login default-ubuntu-1204

Once you've logged in to the system, you can now go ahead and check for the presence of the binary named nginx:

vagrant@default-ubuntu-1204:~$ which nginx

So, Nginx is indeed installed.

However, with kitchen, we no longer need to take the pain of logging in to the system and verifying the installation. We can do this by writing a test case.

We'll make use of bash automated testing system (bats), called for this purpose.

Create a directory using the following command:

$ mkdir -p test/integration/default/bats

Create a new file package test.bats under the bats directory:

#!/usr/bin/env bats

@test "nginx binary is found in PATH"
  run which nginx
  [ "$status" -eq 0 ]

Now, let's run our test using kitchen verify:

$ kitchen verify default-ubuntu-1204
-----> Starting Kitchen (v1.2.1)
-----> Setting up <default-ubuntu-1204>...
Fetching: thor-0.19.0.gem (100%)      
Fetching: busser-0.6.2.gem (100%)      
Successfully installed thor-0.19.0      
Successfully installed busser-0.6.2      
2 gems installed      
-----> Setting up Busser      
       Creating BUSSER_ROOT in /tmp/busser      
       Creating busser binstub      
       Plugin bats installed (version 0.2.0)      
-----> Running postinstall for bats plugin      
Installed Bats to /tmp/busser/vendor/bats/bin/bats      
       Finished setting up <default-ubuntu-1204> (1m41.31s).
-----> Verifying <default-ubuntu-1204>...
       Suite path directory /tmp/busser/suites does not exist, skipping.      
Uploading /tmp/busser/suites/bats/package-test.bats (mode=0644)      
-----> Running bats test suite
✓ nginx binary is found in PATH      
1 test, 0 failures      
       Finished verifying <default-ubuntu-1204> (0m1.03s).
-----> Kitchen is finished. (0m1.51s)

So, we see that our test has successfully passed verification, and we can proudly go ahead and upload our cookbook to the chef-server and trigger a chef-client run on the concerned instance.