Book Image

Chef Essentials

By : John Ewart
Book Image

Chef Essentials

By: John Ewart

Overview of this book

<p>Chef is a configuration management tool that turns IT infrastructure into code. Chef provides tools to manage systems at scale. With this book, you will learn how to use the same tools that companies such as Facebook, Riot Games, and Ancestry.com use to manage and scale their infrastructure.</p> <p>This book takes you on a comprehensive tour of Chef's functionality, ranging from its core features to advanced development. You will be brought up to speed with what's new in Chef and how to set up your own Chef infrastructure for individuals, or small or large teams. Once you have the core components, you will get to grips with bootstrapping hosts to then develop and apply cookbooks. If you want to fully leverage Chef, this book will show you advanced recipes to help you handle new types of data providers and resources. By the end of this book, you will be confident in how to manage your infrastructure, scale using the cloud, and extend the built-in functionality of Chef itself.</p>
Table of Contents (15 chapters)
Chef Essentials
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
Index

Configuring a Chef server


Historically, installing Chef requires manual editing of configuration files, choosing RabbitMQ credentials, installing CouchDB, and a handful of other tasks. Now, with the omnibus installer, all of this is taken care of for you. If you have been following along, your system has the Chef server and all of its dependencies installed on the system in the /opt/chef-server directory.

Included with the installation of the package is a shell script, chef-server-ctl (located at /opt/chef-server/bin), which is responsible for configuring your newly installed Chef server. In order to configure your services, you will need to run it as root because the scripts will need to modify your system in ways that your regular account may not be able to. Initializing the configuration tool is as simple as issuing the following command:

sudo chef-server-ctl reconfigure

Running this script may take a few minutes, and it will produce a lot of output while it is doing its work. While it is running, let's take a few minutes to discuss how it works and what it is doing.

Understanding how chef-server-ctl works

Earlier in this chapter, you were briefly introduced to the chef-solo tool. You saw how it can be used to manage your local server using on-disk recipes and configuration data. The Chef team has leveraged this ability to do just that with the Chef server using chef-solo to bootstrap the server configuration. If you were to look at the code for the /opt/chef-server/bin/chef-server-ctl script, you would see that the last line in the script executes the following command:

/opt/chef-server/embedded/bin/omnibus-ctl chef-server /opt/chef-server/embedded/service/omnibus-ctl $@

If you follow the trail and dig into the omnibus-ctl script, you will find that it is just a wrapper around the omnibus-ctl Ruby gem. Digging into the omnibus-ctl gem, you will see that in the end, the reconfigure command you pass on the command line is a Ruby method that makes the following call:

run_command("chef-solo -c #{base_path}/embedded/cookbooks/solo.rb -j #{base_path}/embedded/cookbooks/dna.json")

This tells us that the Chef omnibus package uses chef-solo to configure itself—a pretty clever trick indeed! You can see just how powerful a tool chef-solo can be, being able to configure and reconfigure the Chef service.

What's happening on my server?

What you will probably notice right away is that a lot of text is being scrolled past in your terminal window. If you were to look at the contents, you would see that it shows you the actions that are being taken by chef-solo to provision your new services. As there is a lot of information going past (thousands of lines), here is a high-level overview of what is happening on your host:

  1. A new user, chef_server, and its corresponding group are being provisioned.

  2. Chef services are being set up, and startup scripts for upstart are being placed in the appropriate system directories. The Run scripts for Chef services are located at /opt/chef-server/sv.

  3. Chef state directories are being created in /var including /var/opt/chef-server and /var/log/chef-server.

  4. RabbitMQ is being configured to store data in /var/opt/chef-server and log the output to /var/log/chef-server as well as its startup scripts in /opt/chef-server/sv/rabbitmq/run.

  5. PostgreSQL is being configured with its data in /var/opt/chef-server/postgresql/data along with a user, opscode-pgsql, to run the service. Some system-level changes to share memory sizes are being set via sysctl to make PostgreSQL work as well as persisted in systctl.conf.

  6. Solr is being set up to work with the configuration and data rooted in /var/opt/chef-server/chef-solr/, with the run script being placed in /opt/chef-server/sv/chef-solr/run.

  7. Chef-expander (the data-indexing service) is being configured for /var/opt/chef-server/chef-expander as its working directory with Solr and RabbitMQ endpoints on the localhost. The run script is located at /opt/chef-server/sv/chef-expander/run.

  8. The Chef bookshelf metadata service is readied in /var/opt/chef-server/bookshelf/ with its run script at /opt/chef-server/sv/bookshelf/run.

  9. Erchef, the Erlang Chef service, is installed and pointed at the local Solr, RabbitMQ, bookshelf, and PostgreSQL services.

  10. The system is then bootstrapped using the bootstrap recipe. This recipe verifies that the system is running (by checking that the http://localhost:8000/_status returns an HTTP 200 response) and installs the SSL certificate for the web-based UI in /etc/chef-server/chef-webui.pem.

  11. The web-based UI configuration files are generated and placed in /var/opt/chef-server/chef-server-webui/.

  12. A copy of nginx to host the web UI is placed in /var/opt/chef-server/nginx, and the initial self-signed SSL certificates as well as the static assets are installed in /var/opt/chef-server/nginx/html.

  13. The Chef API testing framework, chef-pedant, is installed.

  14. Finally, /etc/chef-server/chef-server-running.json is generated with the current configuration settings for your Chef services.

Clearly, there is a lot happening here; if you have any outstanding concerns about what is being done, be sure to read through the output. One of the great things about Chef is that the recipes are just a set of scripts that you can open and view the contents of, and the output shows you what is happening during the execution. Everything it does is transparent and manageable by you.

Verifying that the services are running

Once the configuration of your services is complete, you will want to validate that the required services are running. Again, the chef-server-ctl script will be used, but we will invoke the status subcommand instead of the reconfigure subcommand, as shown in the following code:

user@host:~ $ sudo chef-server-ctl status
run: bookshelf: (pid 3901) 3123s; run: log: (pid 3900) 3123s
run: chef-expander: (pid 3861) 3129s; run: log: (pid 3860) 3129s
run: chef-server-webui: (pid 4053) 3095s; run: log: (pid 4052) 3095s
run: chef-solr: (pid 3819) 3135s; run: log: (pid 3818) 3135s
run: erchef: (pid 4230) 3062s; run: log: (pid 3937) 3117s
run: nginx: (pid 4214) 3064s; run: log: (pid 4213) 3064s
run: postgresql: (pid 3729) 3146s; run: log: (pid 3728) 3146s
run: rabbitmq: (pid 3423) 3172s; run: log: (pid 3422) 3172s

The status subcommand will show you the process ID of each component, how long it has been running for, the PID of the logging process associated with that service, and how long the logging service has been running. For example, we can see that chef-server-webui has a PID of 4053 and has been running for close to an hour, and the logger has a PID of 4052, having been running for just as long as the service.

As you can see, the installation of Chef yields a number of components that will need to be up and running in order to successfully use Chef. You should have the following components running and listening on the following network ports:

Component

What to look for in the process list

Port(s)

Public?

Chef API server

Erchef and nginx

80, 443

Yes

Web management console

chef-server-webui and nginx

80, 443

Yes

Data indexer

chef-expander

N/A

N/A

Solr

java (running start.jar in the Chef directory)

8,983

No

PostgreSQL

postmaster

5,432

No

RabbitMQ

beam.smp running rabbit

5,672

No

Public components need to be made available to any clients, nodes, or end users that expect to use the Chef service over the network. Configuring your infrastructure to ensure that your services are available via the network is outside of the scope of this book as there are a near-infinite number of possible network configurations.

At a higher level, make sure that any firewall devices or packet-filtering systems are not preventing traffic from reaching these services if you see that they are running, but are having difficulties in connecting to them. If any of these services are not running, you will need to consult the logfiles generated by the service to determine what might be preventing them from starting up.