In earlier chapters, we saw a pattern that combined real-time analytics using Storm with batch processing using Hadoop. In this chapter, we will go in the other direction. We will incorporate Storm into an operational system that must respond in real time to end user queries.

Typical applications of Storm focus on a never-ending stream of data. The data is often queued and processed as fast as possible by persistent topologies. The system includes a queue to accommodate varying amounts of load. At times of light load, the queue is empty. During heavy load, the queue will persist the data for eventual processing.

Even the untrained eye will recognize that such a system does not provide true real-time data processing. Storm monitors tuple timeouts, but it is focused on the processing time of tuple(s) after the spout emits the data.

To support real-time scenarios more completely, timeouts and Service Level Agreements (SLA) must be monitored from the reception...