Book Image

Redis 4.x Cookbook

Book Image

Redis 4.x Cookbook

Overview of this book

Redis is considered the world's most popular key-value store database. Its versatility and the wide variety of use cases it enables have made it a popular choice of database for many enterprises. Based on the latest version of Redis, this book provides both step-by-step recipes and relevant the background information required to utilize its features to the fullest. It covers everything from a basic understanding of Redis data types to advanced aspects of Redis high availability, clustering, administration, and troubleshooting. This book will be your great companion to master all aspects of Redis. The book starts off by installing and configuring Redis for you to get started with ease. Moving on, all the data types and features of Redis are introduced in detail. Next, you will learn how to develop applications with Redis in Java, Python, and the Spring Boot web framework. You will also learn replication tasks, which will help you to troubleshoot replication issues. Furthermore, you will learn the steps that need to be undertaken to ensure high availability on your cluster and during production deployment. Toward the end of the book, you will learn the topmost tasks that will help you to troubleshoot your ecosystem efficiently, along with extending Redis by using different modules.
Table of Contents (21 chapters)
Title Page
Dedication
Packt Upsell
Foreword
Contributors
Preface
13
Windows Environment Setup
Index

Understanding the Redis Event Model


Redis, known for its high performance, makes the most of a single thread, non-blocking I/O model to process requests rapidly. Therefore, understanding the event model of Redis is essential. As a taster for readers to understand the model, this recipe first shows an echo server demo program built on the asynchronous event library of Redis (ae library). Then we provide important insights into the event processing model of Redis by analyzing the core snippet of source code in Redis.

This recipe includes a lot of C programming practices. So if you feel unfamiliar with the C language, you can skip this recipe if you wish and it won't bother you too much as you read along.

Getting ready…

This recipe involves source code building and debugging. So you need to finish the Downloading and installing Redis recipe in this chapter first. For better illustration, an IDE that supports the C programming language is needed. The IDE we use here is CLion in Ubuntu Desktop 16.04.3 LTS. While the CLion IDE is not free, a 30-days free trial is enough for us.

You should also prepare the C compiler and development environment. In Ubuntu, you can issue the following command to install the related packages:

$ sudo get update && apt-get install build-essential 

After installation, you should make sure the version of CMake is 3.5 or above:

$ cmake --version 
cmake version 3.5.1 
CMake suite maintained and supported by Kitware (kitware.com/cmake). 

How to do it...

To understand the Redis Event Model, take the following steps:

  1. Untar the source code package of Redis and build some required dependencies manually rather than using CMake:
~$ mkdir coding; cd coding 
~/coding$ tar xzvf redis-4.0.1.tar.gz 
~/coding$ cd redis-4.0.1/deps/ 
~/coding/redis-4.0.1/deps$ make lua linenoise hiredis 

The following screenshot indicates the dependencies have been built successfully:

  1. Download the CLion IDE and untar the package to /redis/coding/:
~/coding$ wget https://download.jetbrains.8686c.com/cpp/CLion-2017.2.2.tar.gz 
~/coding$ tar zxvf CLion-2017.2.2.tar.gz 
  1. Download the demo program package for redis-server building and debugging.
  1. Untar it into the redis-4.0.1 directory:
~/coding$ tar xzvf echodemo.tar.gz -C redis-4.0.1/ 

Make sure the following files, CMakeLists.txt and echodemo exist in the redis-4.0.1 directory:

  1. Log in to the Ubuntu desktop and open a Terminal to start CLion:
~/coding/clion-2017.2.2$ bin/clion.sh 
  1. After accepting the license, you must make sure that the CMake and debugger are both ready as in the following following:

  1. Continue with the default options until you reach the following screenshot:
  1. Click Import Project from Sources and select the redis-4.0.1 sub-directory in the coding directory.
  2. Click the OK button and then select Open Project to open a Redis project.
  3. Choose the Build All configuration in the upper-right corner and click the Run button:

Ignore the error of not specifying an executable target, and click Run.

The following logs show you have managed to build both the echo-server/client demo program and redis-server (my-redis-server in this example):

/redis/coding/clion-2017.2.2/bin/cmake/bin/cmake --build /redis/coding/redis-4.0.1/cmake-build-debug --target all -- -j 6 
Scanning dependencies of target ae 
Scanning dependencies of target my-redis-server 
[  1%] Building C object CMakeFiles/ae.dir/src/zmalloc.c.o 
... 
[ 17%] Building C object CMakeFiles/my-redis-server.dir/src/blocked.c.o 
Scanning dependencies of target echo-server 
... 
[ 25%] Built target echo-server 
[ 26%] Building C object CMakeFiles/my-redis-server.dir/src/config.c.o 
... 
[ 31%] Building C object CMakeFiles/my-redis-server.dir/src/defrag.c.o 
[ 32%] Linking C executable echo-client 
[ 32%] Built target echo-client 
[ 98%] Building C object CMakeFiles/my-redis-server.dir/src/zmalloc.c.o 
[100%] Linking C executable my-redis-server 
[100%] Built target my-redis-server 
  1. In case you want to use the command line in the first place to compile the demo, you can perform the following steps:
/redis/coding/redis-4.0.1$ cmake 
-- The C compiler identification is GNU 5.4.0
...
-- Configuring done
-- Generating done
-- Build files have been written to: /redis/coding/redis-4.0.1
/redis/coding/redis-4.0.1$ make
Scanning dependencies of target my-redis-server
[ 1%] Building C object CMakeFiles/my-redis-server.dir/src/adlist.c.o
[ 2%] Building C object CMakeFiles/my-redis-server.dir/src/ae.c.o
...
[ 85%] Building C object CMakeFiles/my-redis-server.dir/src/zmalloc.c.o
[ 86%] Linking C executable my-redis-server
[ 86%] Built target my-redis-server
Scanning dependencies of target ae
...
[ 92%] Built target ae
Scanning dependencies of target echo-server
...
[ 96%] Built target echo-server
Scanning dependencies of target echo-client
...
[100%] Built target echo-client
  1. You can find echo-server, echo-client, and my-redis-server in the cmake-build-debug directory under redis-4.0.1:
  1. Choose the echo-server configuration in the upper-right corner, and click the right arrow button to get it running:
  1. Open a Terminal and then connect to the echo-server using nc (Netcat):
~/coding/redis-4.0.1/cmake-build-debug$ nc 127.0.0.1 8000 
  1. The message Hello Client! will be printed out when the server has started successfully:

The server logs the connection:

  1. Type Hello, please echo! in nc and press Enter to send it out. The same message will be echoed back:

On the server side, it logs the data, which will be sent back to the client later:

  1. Open another Terminal and then make another connection to the echo-server. A similar result will be obtained on both nc and server side, as shown in the following screenshot:

  1. You can debug this server if you wish:
  1. You can build and debug redis-server (called my-redis-server in this example) to dig into the source code in almost the same way as the echo-server example. The only thing you have to change is to select the my-redis-server run/debug configuration profile:

How it works...

As mentioned earlier, Redis takes great advantage of the non-blocking, multiplexing I/O model in its main processing single thread, although there are some circumstances where Redis spawns threads or child processes to perform certain tasks.

Redis contains a simple but powerful asynchronous event library called ae to wrap different operating system's polling facilities, such as epoll, kqueue, select, and so on.

So what's a polling facility of an operating system? Let's take a real-life scenario to illustrate it. Imagine you have ordered five dishes in a restaurant. You have to fetch your dishes by yourself at a waiting window, and you want to get them as soon as possible once the dishes get done because you are hungry. You may have three strategies for this scenario:

  • You walk to the waiting window all by yourself from time to time in a short period to check whether each dish in the order list is ready.
  • You hire five people. Each person walks to the waiting window for you to check whether one of the dishes you ordered is ready.
  • You just sit at the table, waiting for the notification. The restaurant provides a dish-ready notification service for free which means that the waiter will tell you which dish is ready, once a dish gets done. When you get the notification, you fetch it by yourself.

Considering the time and efforts it takes, the third option is the best one, obviously.

The polling facility of an operating system works in a similarly way as the third option does. For the simplicity of this section, we only take the epoll API in Linux as an example. First, you can call epoll_create to tell the kernel that you would like to use epoll. Then, you call epoll_ctl to tell the kernel the file descriptors (FD) and what type of event you're interested in when an update occurs. After that, epoll_wait gets called to wait for certain events of the FDs you set in epoll_ctl. The kernel will send a notification to you when the FDs get updated. The only thing you have to do is to create handlers for certain events.

The whole multiplexing process is shown as follows:

I/O multiplexing model

The ae library in Redis basically follows the preceding procedure to process the requests. In the echo-server example, we create an event loop by calling aeCreateEventLoop firstly. Then a TCP server is built via anetTcpServer for network binding and listening. We call anetNonBlock to set the non-block I/O action for this socket FD. After that, we specify the acceptance event handler acceptProc for the socket FD using the event loop created in aeCreateEventLoop. Once a TCP connection is established, the server will trigger the action in acceptProc. In acceptProc, we use anetTcpAccept to accept the connection request and register readable events of the socket FD for readProc. Then readProc gets called, in which we read the data sent to the server and register the writable event of the socket FD. The event loop then receives the writable event to fire the writeProc to send back the data to the socket client.

Redis works in pretty much the same way as this echo-server does. In the main function of server.c, aeCreateEventLoop, anetTcpServer, and anetNonBlock get called in the initServer method to initialize the server:

The acceptance handler is also set in the initServer method, as the following screenshot shows:

Once the server is initialized, the aeMain method gets called:

In the aeMain method, asProcessEvents is called to process the events continuously:

There's more...

It's clear that no thread or sub-process spawns or interacts in the polling process. Therefore, the key benefit of this model is that it's a light context switch I/O model, so that it's not costly for context switching. A number of limitations need to be considered for this processing model. The most common problem you may encounter is latency. In the polling model, Redis won't process any other commands until the one being processed finishes. So keep in mind from now on that an unexpected latency will be the first headache for you when using Redis.

Other polling methods, such as poll, select, and so on, are not discussed here due to the limited area. If you are working on platforms other than Linux, you can debug the source code to learn more about the polling facility on that platform.

See also

  • Refer to the Identifying slow operations/queries using SLOWLOG and Troubleshooting latency issues, sections in Chapter 10, Troubleshooting Redis, for more detailed approaches to Redis latency troubleshooting
  • Refer to Appendix, Windows Environment Setup if you want to build and debug the source code in Windows
  • You can also further refer to the following links:
    • https://redis.io/topics/internals-eventlib offers you a deep insight into the event library
    • https://redis.io/topics/internals-rediseventlib describes the details of the Redis event library
  • If you are working on operating systems other than Ubuntu, you can refer to https://cmake.org/install/ to install CMake