In this chapter, we gained a deeper appreciation for the Trident State API. We created a direct implementation of the State and StateUpdater interfaces instead of relying on default implementations. Specifically, we implemented these interfaces to bridge the gap between a transactional spout and a non-transactional system, namely Druid. Although it is impossible to establish exactly-once semantics into a non-transactional store, we put mechanisms in place to alert when the system encounters issues. Ostensibly, upon failure we could then use a batch processing mechanism to reconstruct any suspect aggregation segments.

For future investigation, it would be beneficial to establish an idempotent interface between Storm and Druid. To do this, we could publish a single segment for each batch within Storm. Since segment propagation is atomic within Druid, this would give us a mechanism to commit each batch atomically to Druid. Additionally, batches could then be processed in parallel, improving...