Book Image

Mastering RethinkDB

By : Shaikh
Book Image

Mastering RethinkDB

By: Shaikh

Overview of this book

RethinkDB has a lot of cool things to be excited about: ReQL (its readable,highly-functional syntax), cluster management, primitives for 21st century applications, and change-feeds. This book starts with a brief overview of the RethinkDB architecture and data modeling, and coverage of the advanced ReQL queries to work with JSON documents. Then, you will quickly jump to implementing these concepts in real-world scenarios, by building real-time applications on polling, data synchronization, share market, and the geospatial domain using RethinkDB and Node.js. You will also see how to tweak RethinkDB's capabilities to ensure faster data processing by exploring the sharding and replication techniques in depth. Then, we will take you through the more advanced administration tasks as well as show you the various deployment techniques using PaaS, Docker, and Compose. By the time you have finished reading this book, you would have taken your knowledge of RethinkDB to the next level, and will be able to use the concepts in RethinkDB to develop efficient, real-time applications with ease.
Table of Contents (11 chapters)

Constraints and limitation in RethinkDB

We have covered various architectural features and the data model of RethinkDB. Let's look over some of the constraints of RethinkDB that you need to take into account while architecting your data store.

RethinkDB divides the limitation into hard and soft limitations. The hard limitations are as follows:

  • The number of databases has no hard limit
  • There is a limit on shard creation, which is a maximum of 64 shards
  • Creation of tables inside the database has no hard limit
  • Storage size of a single document has no hard limit (however, it is recommended to keep it under 16 MB for performance reasons)
  • The maximum possible size of a query is 64 MB

RethinkDB also has some memory limitation, as follows:

  • An empty table will need up to 4 MB
  • Each table, after population of documents, requires at least 10 MB of disk space on each server wherever it is replicated in the cluster
  • Each table consumes 8 MB of RAM on each server

RethinkDB, in order to keep performance high, stores some data in the RAM. There are basically three sources of usage of RAM by RethinkDB:

  • Metadata
  • Page cache
  • Running queries and background processes

RethinkDB stores metadata of tables in the main memory in order to ensure fast read access. Every table consumes around 8 MB per server for the metadata. RethinkDB organizes the data into blocks, with size ranging from 512 bytes to 4 KB. Out of these blocks, approximately 10 to 26 bytes per block are kept in memory.

Page cache is a very important aspect of performance. It is basically used to store very frequently accessed data in the RAM rather than reading it from disk (except in the case of direct I/O, where the page cache is in the application buffer than RAM). RethinkDB uses this formula to calculate the size of the cache:

Cache size = available memory - 1024 MB / 2

If the cache size is less than 1224 MB, then RethinkDB set the size of page cache to 100 MB. This is why it is recommended to have at least 2 GB of RAM allocated for RethinkDB processes.

You can also change the size of the page cache when you start the server or later, using configuration files.

Every database uses some memory to store the results of ongoing running queries. Since queries differ, in general, there is no exact estimate about memory usage by running queries; however, a rough estimate is between 1 MB and 20 MB, including background processes such as transferring data between nodes, voting processes, and so on.