Book Image

Learning RabbitMQ

By : Martin Toshev
Book Image

Learning RabbitMQ

By: Martin Toshev

Overview of this book

RabbitMQ is Open Source Message Queuing software based on the Advanced Message Queue Protocol Standard written in the Erlang Language. RabbitMQ is an ideal candidate for large-scale projects ranging from e-commerce and finance to Big Data and social networking because of its ease of use and high performance. Managing RabbitMQ in such a dynamic environment can be a challenging task that requires a good understanding not only of how to work properly with the message broker but also of its best practices and pitfalls. Learning RabbitMQ starts with a concise description of messaging solutions and patterns, then moves on to concrete practical scenarios for publishing and subscribing to the broker along with basic administration. This knowledge is further expanded by exploring how to establish clustering and high availability at the level of the message broker and how to integrate RabbitMQ with a number of technologies such as Spring, and enterprise service bus solutions such as MuleESB and WSO2. We will look at advanced topics such as performance tuning, secure messaging, and the internals of RabbitMQ. Finally we will work through case-studies so that we can see RabbitMQ in action and, if something goes wrong, we'll learn to resolve it in the Troubleshooting section.
Table of Contents (18 chapters)
Learning RabbitMQ
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
Index

RabbitMQ repositories


The RabbitMQ repositories are located in GitHub—the message server, the plugins that come with the RabbitMQ installation, and the additional tools—all of them in one place.

Getting the sources

The RabbitMQ repositories are located in GitHub at https://github.com/rabbitmq. You have to first install Git in order to be able to check the RabbitMQ sources and build the various components of the broker. In order to clone the RabbitMQ server repository, you can navigate to a proper directory and execute it from your Git command client:

git clone https://github.com/rabbitmq/rabbitmq-server rabbitmq-server

Building the RabbitMQ server

After you have cloned the RabbitMQ server repository, you can build the message broker using the GNU Make utility from the root source directory (depending on the operating system of your choice, you may have to download and install either GNU make or a port of the utility for the particular operating system). It is easier if you build RabbitMQ under a Linux distribution such as Ubuntu (we are using Ubuntu 12.04 for the sample build). However, before you are able to build the message server, you need to install the libxslt and xsltproc libraries that provide utilities for XSLT (Extensible Stylesheet Language Transformations) processing that is used by the RabbitMQ server and the erlang-nox and erlang-dev packages that provide additional Erlang tools used by RabbitMQ:

sudo apt-get install libxml2-dev libxslt1-dev xsltproc erlang-nox erlang-dev

You also need to install OpenSSL on your distribution, in case it isn't already installed. Make sure that you are also using a proper version of Erlang, Git, and Python for your operating system—otherwise your build may fail at some point—for this particular example, we are using Erlang/OTP 18 [erts-7.1], Git version 2.6.3, and Python 2.7.1. In order to build the message server, go to the local RabbitMQ server repository and use the make utility, as follows:

cd rabbitmq-server make

The above command calls the default target as defined in Makefile that supplies the build targets for RabbitMQ, it uses the erlang.mk utility that provides utilities to build Erlang applications using make. However, the erlang.mk utility has limited support for Windows (at the time of writing, MSYS2 support was just introduced and there is still no support for Cygwin). You can download MSYS2 from https://msys2.github.io/. Then, install the make and diffutils packages as follows:

pacman -S make
pacman -S diffutils

You also need to download xsltproc (32 bit or 64 bit) — you can use the following link:

http://www.zlatkovic.com/pub/libxml/

Copy the contents of the bin directory from the ZIP file to the usr\bin directory of MSYS2 (you may need to download and extract additional libraries; try to run the xsltproc tool from the command line in order to make sure it runs fine. During the build of the server, you may receive the following error:

error: rabbitmq-components.mk must be updated!

In order to provide a workaround for it, open the rabbitmq-components.mk file and go to the following code snippet:

$(verbose) cmp -s rabbitmq-components.mk \
$(UPSTREAM_RMQ_COMPONENTS_MK) || \
(echo $(UPSTREAM_RMQ_COMPONENTS_MK))
(echo "error: rabbitmq-components.mk must be updated!" 1>&2; \
 false)

Change it to the following (in order to skip the validation due to different end-of-line characters on Unix and Windows):

check-rabbitmq-components.mk:true

The reason for doing this is that the cmp utility by RabbitMQ, in order to verify the contents of the rabbitmq-components.mk file, does not respect line endings and the check fails (since the already existing rabbitmq-components.mk file from the repository has Linux-style line endings, while the generated one has Windows-style line endings).

After you have built the RabbitMQ server, you will notice that the Erlang source files from the src directory are compiled to beam files in the ebin directory. You can now run an instance of the RabbitMQ server that uses a temporary Mnesia database using the run target:

make run-broker

This is particularly useful if you are developing plugins for RabbitMQ and want to test them. If you want to rebuild the server, you can first execute the clean target in order to remove the artifacts from the old build:

make clean

You can build distributable RabbitMQ packages for all the supported platforms by running the following:

make packages

You can also build a RabbitMQ package for a particular platform only. In order to build a Debian package, you can run:

make package-deb

To build a package for Windows, run the following:

make package-windows

You may need to install additional packages along the process, as follows:

sudo apt-get install tofrodos
sudo apt-get install xmlto
sudo apt-get install elinks

To support the analysis of the RabbitMQ sources and development of RabbitMQ plug-ins, you can use various utilities. The Erlide is an Eclipse plugin that provides Erlang development tools in the IDE: http://erlide.org/. You can also use the xref utility that is provided as part of the OTP toolset in order to analyze module dependencies. For example, after you have compiled the RabbitMQ sources with the make utility, you can use xref to see the modules in which a specified module depends (go to the ebin directory with the compiled beam files for the RabbitMQ server). For example, the rabbitmq_sup module, which creates a process supervisor for other processes that are running in the broker, does not have any dependencies (the undefined array from the result is empty — it includes information about the used modules), as shown in the following:

xref:m(rabbit_sup).

The output from the preceding invocation is as follows:

[{deprecated,[]},{undefined,[]},{unused,[]}]

If we do the same for the rabbit module, which boots the RabbitMQ server, we will see a number of dependencies:

xref:m(rabbit).

The output from the previous invocation looks like the following:

[{deprecated,[]},
 {undefined,[{{rabbit,alarms,0},{rabbit_misc,const,1}},
             {{rabbit,alarms,0},{rabbit_misc,with_exit_handler,2}},
             {{rabbit,boot_error,2},{rabbit_misc,format,2}},
             {{rabbit,boot_error,2},{rabbit_nodes,diagnostics,1}},
             {{rabbit,erts_version_check,0},
              {rabbit_misc,version_compare,3}},
             {{rabbit,force_event_refresh,1},
              {rabbit_amqqueue,force_event_refresh,1}},
               …
{unused,[]}]

Another useful utility is the module_info built-in function that allows you to retrieve detailed information about a module. The following example retrieves the information about the rabbitmq_sup module:

rabbit_sup:module_info().

The output of the preceding information about the module that includes the information about exported functions, module attributes, and other information for the module is as follows:

[{module,rabbit_sup},
 {exports,[{start_child,1},
           {start_child,2},
           {start_child,3},
           {start_supervisor_child,1},
           {start_supervisor_child,3},
           {start_restartable_child,1},
           {start_delayed_restartable_child,1},
           {start_delayed_restartable_child,2},
           {stop_child,1},
           {init,1},
           {module_info,0},
           {module_info,1},
           {start_restartable_child,2},
           {start_supervisor_child,2},
           {start_link,0}]},
 {attributes,[{vsn,[57416127534960432714786320802993587506]},
              {behaviour,[supervisor]}]},
 {compile,[{options,[{d,use_specs},
                     {d,'INSTR_MOD',gm},
                     {outdir,"/home/openjdk/rabbitmq-server/ebin"},
                     {i,"/home/openjdk/rabbitmq-server/deps/rabbit_common/include"},
                     {i,"/home/openjdk/rabbitmq-server/include"},
                     warn_obsolete_guard,warn_shadow_vars,
                     warn_export_vars,debug_info,
                     warnings_as_errors]},
           {version,"6.0.1"},
           {time,{2015,11,14,13,35,5}},
           {source,"/home/openjdk/rabbitmq-server/src/rabbit_sup.erl"}]},
 {native,false},
 {md5,<<"+1\361\245v\327TcBt\275\300\250\326\3052">>}]

For detailed information on the output of module_info, you can refer to Erlang User's Guide.