Book Image

Mastering Redis

By : Jeremy Nelson
Book Image

Mastering Redis

By: Jeremy Nelson

Overview of this book

Redis is the most popular, open-source, key value data structure server that provides a wide range of capabilities on which multiple platforms can be be built. Its fast and flexible data structures give your existing applications an edge in the development environment. This book is a practical guide which aims to help you deep dive into the world of Redis data structure to exploit its excellent features. We start our journey by understanding the need of Redis in brief, followed by an explanation of Advanced key management. Next, you will learn about design patterns, best practices for using Redis in DevOps environment and Docker containerization paradigm in detail. After this, you will understand the concept of scaling with Redis cluster and Redis Sentinel , followed by a through explanation of incorporating Redis with NoSQL technologies such as Elasticsearch and MongoDB. At the end of this section, you will be able to develop competent applications using these technologies. You will then explore the message queuing and task management features of Redis and will be able to implement them in your applications. Finally, you will learn how Redis can be used to build real-time data analytic dashboards, for different disparate data streams.
Table of Contents (18 chapters)
Mastering Redis
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
Index

Preface

The intention of Mastering Redis is to build upon your basic knowledge of Redis through two ways; provide the deeper meaning of the context and theory behind Redis and its technologies, and increase your practical day-to-day skills with Redis. The Mastering in this book's title implies an ongoing process and not an end destination. What is exciting about Redis is its ongoing and public evolution into the powerful data manipulation and storage technology of today.

The philosophy behind Redis

Salvatore Sanfilippo has, over the lifespan of the project, articulated a distinct view and opinion about the direction and functionality of Redis. In a January 2015 blog post about benchmarking Redis against other databases, Sanfilippo states "I don't want to convince developers to adopt Redis. We just do our best in order to provide a suitable product, and we are happy if people can get work done with it. That's where my marketing wishes end." Sanfilippo and a small core group of Redis developers follow the successful open source governance model of the "benevolent dictator for life" (BDL), where a single person is the ultimate arbitrator of what is committed into the Redis code base. The success of the BDL model, evidenced by open source projects such as Linux kernel development and the Python programming language, is replicated in Redis with Sanfilippo as its primary developer and maintainer.

The BDL model failure modes can be catastrophic if the dictator abandons the project, or worse, is incapacitated through illness or death. Another significant problem that has emerged particularly with Redis is when potential contributors submit pull requests and action on their pull requests is delayed, or more often, ignored. To be fair, the volume of changes that must be examined, tested, and merged into the main code base can be substantial and requires a passionate and dedicated gatekeeper. Linus Torvalds, the initial creator and current BDL for the Linux kernel project, has seen his role evolve more into merging code contributed by others and providing vision and leadership for Linux than writing code himself. Sanfilippo, while acknowledging this problem in a thread on the main Redis e-mail distribution, gives two main reasons for continuing with the current BDL model for Redis:

  • A consistent vision for the project's development and future directions

  • Accountability for any new or merged changes

Sanfilippo's vision of Redis, as an easy-to-configure, small-memory-footprint (for itself and NOT for its datasets!) and reliable key-value data store has been crucial to the continued rise in Redis's popularity among developers and organizations. His vision does cause tension, especially when new features for Redis are proposed, such as expiring specific sub-values in a hash or offering loadable modules for optional functionality, and these features are rejected for inclusion into Redis. Sanfilippo's desire to keep Redis small and focused on being a memory-only database drives his decisions and development practice.

In 2011 blog post, he elucidated his vision for Redis in a seven-point manifesto for Redis and the Redis development process. Briefly, here are the seven points:

  1. A DSL for Abstract Data Types: Redis is a Domain-Specific Language (DSL) for representing and using abstract data structures. These data structures include both the operations (Redis commands) as well as the memory efficiency and time complexity of storing and manipulating those data structures with the associated Redis commands.

  2. Memory storage is the #1: By storing all of the data in a computer's RAM, Redis's performance across different systems is more consistent, the various algorithms used to implement these data structures run in a more predictable fashion, and more complex data types such as sorted sets are easier to implement in an in-memory database.

  3. Fundamental data structures for a fundamental API: Redis implements a fundamental API for its fundamental data structures. This API, made up of Redis commands and corresponding data structures, tries to intelligibly resemble the data structures the API reads and writes to the computer's memory. Following this design, the Redis API builds more complex operations into the API by building from simpler operations on data structures in the API.

  4. Code is like a poem: The most elusive of the seven points in this manifesto. Sanfilippo gives his aesthetic preference for code that fits into a larger narrative of the entire Redis project. His point is that Redis's coding style and approach are geared for humans to construct a narrative. So, inclusion of third-party code depends in part on how well the code fits into the large narrative of Redis and Redis's source code.

  5. We're against complexity: Complexity in code is to be avoided. Given a choice to build a small feature with a lot of implementing of code or to forgo the functionality, Redis will take the latter route and forgo the extra complexity and overhead of adding complexity to the code base.

  6. Two levels of API: Redis starts with a subset of its API to run in a distributed manner and a larger, more functionality-rich API to support multikey operations. This separation allows significant features such as the Redis master-slave and Redis cluster modes of operation.

  7. We optimize for joy: An emotional appeal and very intelligent statement, for developers and operators of technology in general, the thrill of tuning technology to solve difficult and complex problems does elicit feelings of happiness and excitement about the future possibilities of Redis.

What this book covers

As you read Mastering Redis, two themes will emerge that parallel the development/operations dualism of the popular and trendy operations and processes, commonly known as DevOps. To help guide your approach to the material contained in the chapters, each chapter's topics will be identified as either software development or system operations focused. Due to the increasingly blurred line between the two, getting a topical understanding of the topics in each trend increases your and your team's abilities to quickly and efficiently develop and deploy Redis solutions for your project or as a piece of your technological infrastructure requirements.

In the following diagram, each chapter's horizontal position visually represents whether the topics weigh towards software development or systems operations:

DevOps Chapter Tracks

Chapter 1, Why Redis?, introduces the Redis development philosophy as articulated by Salvatore Sanfilippo, the founder and primary maintainer of Redis.

Chapter 2, Advanced Key Management and Data Structures, builds upon your basic knowledge of Redis by expanding and explaining Redis data structures and key management, including the important topic of constructing meaningful and expressive key schemas for your applications.

Chapter 3, Managing RAM – Tips and Techniques for Redis Memory Management, looks at the various options Redis provides to optimize the memory usage in your applications including Redis support for various caching and key eviction strategies based on Less Recently Used (LRU) implementations in Redis.

Chapter 4, Programming Redis Part One – Redis Core, Clients, and Languages, is an advanced topic on programming applications. This chapter starts with an overview of Redis's core C programming language implementation and includes an in-depth examination of selected C code snippets to deepen your knowledge of Redis. It continues with how to use three different Redis clients, with short programming exercises in Python, Node.js, and Haskell.

Chapter 5, Programming Redis Part Two – Lua Scripting, Administration, and DevOps, is an advanced topic on programming applications. It starts with an overview of Redis server-side Lua scripting and how to use Lua more effectively with Redis. The chapter next expands on a few popular programming design patterns with Redis, with specific examples of how different people and companies have used these patterns in their operations. This chapter ends with how Redis is used in typical DevOps scenarios from the perspective of a software developer.

Chapter 6, Scaling with Redis Cluster and Sentinel, explores two relatively recent additions to Redis—Redis Cluster and Redis Sentinel. Redis Sentinel is a special high-availability mode for monitoring the health of masters and slaves, along with the ability to switch if a failure occurs in any master or slave Redis instance. Redis Cluster, mentioned previously, is now a production-ready way to store large amounts of data that may be too big to fit into the memory of a single machine, by running multiple Redis instances through key sharding. While these topics have more of an operational focus, engineering solutions with Redis should, at the minimum, know the benefits and limitations of how to use Redis Cluster.

Chapter 7, Redis and Complementary NoSQL Technologies, starts with the recognition that for most organizations, their information technology stack includes a heterogeneous mixture of different types of data and processing solutions. Redis is an ideal way to extend the functionality of other NoSQL data storages options, and in this chapter, we'll see how Redis can be used with MongoDB, ElasticSearch, and Fedora Digital Repository. This chapter should be of interest to both developers and system administrators who may need to develop and support complex business requirements with multiple solutions.

Chapter 8, Docker Containers and Cloud Deployments, shows how using Redis as in Docker containers and images can simplify management and improve security and reliability of your Redis solutions. Docker is an open source container technology for applications that is rapidly being adopted by many enterprises. Building upon Docker with Redis, we'll then examine specific challenges of using Redis on the most popular computing cloud providers starting the largest and most established, Amazon Web Services, followed by Google's Compute Engine and Microsoft Azure, with special attention to other cloud service providers such as Rackspace and Digital Ocean. We'll finish the chapter by examining Redis's offerings of specialized cloud services that focus on hosting and managing your Redis instances.

Chapter 9, Task Management and Messaging Queuing, begins with an in-depth exploration of Redis Pub/Sub commands. This involves first looking at various examples of how publishers and consumers can communicate between different processes, programs, Redis clients, operating systems, and remote computers. Further in the chapter, we'll expand upon Redis Pub/Sub and look more generally at using Redis as a messaging queue between different layers in an enterprise computing ecosystem. This chapter ends by wrapping up all the concepts through a detailed example of using Redis with Celery as task management and a messaging queue with Pub/Sub support.

Chapter 10, Measuring and Managing Information Streams, builds upon the previous chapter's concepts to show how Redis is be used as a real-time data aggregator for disparate data streams of various technology systems used within an organization. We'll then examine the Redis security model and new security features with the latest version Redis. A web-based, operational dashboard will visualize the incoming data flows into Redis using our knowledge of Redis clients. Next, we'll show how to apply machine learning algorithms, such as Naive Bayes, to these Redis-based information flows to provide a richer snapshot and deepen your understanding of the operations occurring within an organization or department.

Appendix, Sources, acknowledges the source of extracts used in the chapters and presents links chapter-wise for further reading.

Earn your Mastering Redis Open Badge

The Mozilla Foundation—the same open source organization that sponsors the development of the Firefox web browser—started a project called Open Badges that allows organizations to create and then issue portable and non-proprietary badges to individuals to signal accomplishments:

At the Mastering Redis website, you have the opportunity to signal to your current and potential employers your increased knowledge and skills with Redis by taking a series of online quizzes and earning your Mastering Redis Open Badge. Your Open Badge can be shared through popular social networking sites such as Facebook, Twitter, or LinkedIn.

The Mastering Redis Open Badge is free to readers who have purchased the book. However, for readers who don't own a copy, you can still earn your Open Badge at the book's website for a nominal fee. The opportunity to connect with other badge earners, learning from their experiences with Redis while sharing your own stories and knowledge and thus encourages learning long after you have finished reading Mastering Redis. Our hope is that this book can immediately help your understanding of Redis and that by earning your Open Badge, you can document this professional achievement.

What you need for this book

Redis is intended to be run under a POSIX-based environment such as Linux or Mac OX with a modern C++ compiler. Microsoft Windows versions of Redis are available but not officially supported. Please see the Windows section at http://redis.io/download for more information. Examples in this book also use Python 3.5 with the Redis Python client (https://github.com/andymccurdy/redis-py), Lua, and Node.js with the Redis Node.js client (https://github.com/NodeRedis/node_redis).

Who this book is for

If you are a web developer with a basic understanding of the MEAN stack, experience in developing applications with JavaScript, and basic experience with NoSQL databases, then this book is for you.

Conventions

In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "First, when you call a timeout with the EXPIRE command on a key, the timeout can only be cleared if you delete the key or replace the key."

A block of code is set as follows:

defcreate_tea(datastore, name, time, size):
  # Increment and save global counter
  tea_counter = datastore.incr("global/teas")
  tea_key = "tea/{}".format(tea_counter)
 datastore.hmset(tea_key,
      {"name": name,
      "brew-time": time,
      "box-size": size})
  return tea_key

Any command-line input or output is written as follows:

127.0.0.1:6379> LATENCY HISTORY command
1) 1) (integer) 1433877379
   2) (integer) 1000
2) 1) (integer) 1433877394
   2) (integer) 250

Note

Warnings or important notes appear in a box like this.

Tip

Tips and tricks appear like this.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.

To send us general feedback, simply e-mail , and mention the book's title in the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

You can download the code files by following these steps:

  1. Log in or register to our website using your e-mail address and password.

  2. Hover the mouse pointer on the SUPPORT tab at the top.

  3. Click on Code Downloads & Errata.

  4. Enter the name of the book in the Search box.

  5. Select the book for which you're looking to download the code files.

  6. Choose from the drop-down menu where you purchased this book from.

  7. Click on Code Download.

Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:

  • WinRAR / 7-Zip for Windows

  • Zipeg / iZip / UnRarX for Mac

  • 7-Zip / PeaZip for Linux

The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Mastering-Redis. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Downloading the color images of this book

We also provide you with a PDF file that has color images of the screenshots/diagrams used in this book. The color images will help you better understand the changes in the output. You can download this file from https://www.packtpub.com/sites/default/files/downloads/MasteringRedis_ColorImages.pdf.

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.

To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.

Piracy

Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.

Please contact us at with a link to the suspected pirated material.

We appreciate your help in protecting our authors and our ability to bring you valuable content.

Questions

If you have a problem with any aspect of this book, you can contact us at , and we will do our best to address the problem.