In this chapter, we saw that pairing a batch processing mechanism with a real-time processing engine such as Storm provides a more complete and robust overall solution.

We examined an approach to implementing a Lambda architecture. Such an approach delivers real-time analytics supported by a batch processing system retroactively correcting the analytics. Additionally, we saw how to configure a multidata center system architecture to isolate the offline processing from the transactional system while also providing continuous availability and fault tolerance via distributed storage.

The chapter also included an introduction to Hadoop, using Druid's implementation as an example.

In the next chapter, we will take an existing batch process that leverages Pig and Hadoop and demonstrate what it takes to convert that into a real-time system. At the same time, we will demonstrate how to deploy Storm onto the Hadoop infrastructure using Storm-YARN.