Book Image

Extending SaltStack

Book Image

Extending SaltStack

Overview of this book

Salt already ships with a very powerful set of tools, but that doesn't mean that they all suit your needs perfectly. By adding your own modules and enhancing existing ones, you can bring the functionality that you need to increase your productivity. Extending SaltStack follows a tutorial-based approach to explain different types of modules, from fundamentals to complete and full-functioning modules. Starting with the Loader system that drives Salt, this book will guide you through the most common types of modules. First you will learn how to write execution modules. Then you will extend the configuration using the grain, pillar, and SDB modules. Next up will be state modules and then the renderers that can be used with them. This will be followed with returner and output modules, which increase your options to manage return data. After that, there will be modules for external file servers, clouds, beacons, and finally external authentication and wheel modules to manage the master. With this guide in hand, you will be prepared to create, troubleshoot, and manage the most common types of Salt modules and take your infrastructure to new heights!
Table of Contents (21 chapters)
Extending SaltStack
About the Author
About the Reviewer

Using plugins

As Salt was originally designed as a backbone that other software could use to communicate, its earliest purpose was to collect information from a large cluster of both physical and virtual machines, and return that data either to the user or to a database. Various programs, such as ps, du, and netstat, were used to collect that information. Because of that, each program was wrapped with a plugin, which contained various functions to call those programs, and parse the return data.

Those plugins were originally called modules. Later, when other types of module were added to Salt, the original modules began to be referred to as execution modules. This is because the execution modules would do the heavy lifting, and other types of module would generally wrap around them and extend their functionality.

Loading modules

Like many data centers, the one that Salt was created in had various servers that used different software packages to perform their work. One server would be running Nginx, while another would be running DNSMasq. It wouldn't make sense to enable the nginx module on the DHCP server, or a dnsmasq module on the web server. A number of popular programs solve this by allowing the user to configure which plugins will be loaded before starting the service.

Salt had a different way of handling plugins. In a large infrastructure, individual configuration of servers can be costly in terms of time. And as configuration management was added to Salt, a core belief grew that configuration management platforms should require as little configuration themselves as possible. What is the point of using such a suite to save time if so much time is required to get it going in the first place?

This is how the loader system came to be. Salt would always ship with a full set of modules, and Salt would automatically detect modules that would be available, and dynamically load them.

Execution modules are a type of plugin that performs most of the heavy lifting inside of Salt. These were the first to use the loader system, and for a short time there was no other type of module. As the functionality of Salt increased, it quickly became evident that other types of module would be needed. For instance, return output was originally just printed to the console. Then the output was changed to be easier to handle from shell scripts. Then the outputter system was added, so that output could be displayed in JSON, YAML, Python's pprint, and any other format that might be useful.

Standard modules

In the beginning, there were some types of module that would always be loaded. The first of these was the test module, which required nothing more than Salt's own dependencies; in particular, it would only require Python.

Other modules were also designed for general use, requiring no more than Salt's own dependencies. The file module would perform various file-based operations. The useradd module would wrap the standard Unix useradd program. This was fine, so long as Salt was only used on Unix-like platforms. When users started running Salt on Windows, where those utilities were not readily available, things changed. This is where virtual modules really started to shine.

Virtual modules

Supporting Salt on various platforms, such as both Unix-like and Windows, presents the same problem as whether or not to make the nginx module available: if that platform is installed and available, make the module available. Otherwise, don't. Salt handles the availability problem by implementing virtual modules.

The idea behind a virtual module is that it will contain a piece of code that will detect whether or not its dependencies are met, and if so, the module will be loaded and made available to Salt on that system. We'll get into the details of actually doing this in Chapter 2, Writing Execution Modules.

Lazy loading modules

In the beginning, if a module was detected as being loadable, then it would be loaded as the Salt service was started. A number of modules may be loaded for a particular system, which the administrator never intends to use. It may be nice to have them, but in some cases it's better to only load them when they're needed.

When the Salt service starts, the lazy loader will detect which modules may be used on a particular system, but it won't immediately load them into memory. Once a particular module is called, Salt will load it on demand, and then keep it in memory. On a system that typically only uses a small handful of modules, this can result in a much smaller footprint than before.