With the design in place, we can turn our attention to the implementation. The sequence diagram for the implementation is shown as follows:
The preceding diagram implements the state machine shown in the design. Once the real-time server is started, Druid polls the StormFirehose
object using the hasMore()
method. The contract with Druid specifies that the Firehose
object's implementation should block until data is available. While Druid is polling and the Firehose
object is blocking, Storm delivers tuples into the DruidState
object's
message buffer. When the batch is complete, Storm calls the commit()
method on the DruidState
object. At that point, the PartitionStatus is updated. The partition is put in progress and the implementation unblocks the StormFirehose
object.
Druid begins pulling data from the StormFirehose
object via the nextRow()
method. When the StormFirehose
object exhausts the contents of the partition, it places the partition in limbo, and releases...