The explosive growth in big data is not only in terms of the volume being generated but also in terms of the variety and speed at which the results have to be generated in order to be meaningful. Thus, the velocity of the data and computation has forced developers toward real-time stream processing frameworks, and at the same time, the variety and unstructured nature of data has led to the NoSQL movement.
With the rise of the Internet of Things (IoT), sensors, social media, machine transactions, monitoring data, and so on are being produced at a very large scale and velocity. The insights provided by this data can be very valuable, but the analysis and the data itself do not make sense if results are produced with a delay, or analysis is done on the stale data. In the previous chapters, we looked at how large amounts of data can be processed using Hadoop and Spark. These traditional tools are very well suited for batch or offline analysis...