You must be wondering why we need to talk about high volumes in this chapter, considering how aspects such as high volumes, performance, and scalability are in the genes of Kafka architecture. Well, you are thinking in the right direction, but certain parameters need to be tuned to manage Kafka latency and throughput requirements.
Moreover, you have to choose the right set of hardware and perform appropriate capacity planning. Therefore, we thought that it is better to discuss it. In a nutshell, when we talk about high volumes in Kafka, you have to think of following aspects:
- High volume of writes or high message writing throughput
- High volumes of reads or high message reading throughput
- High volume of replication rate
- High disk flush or I/O
Let's look at some of the components that you should consider for high volumes in Kafka.
...