Now, we will create a Storm topology that will consume messages from a Kafka topic, word_topic, and aggregate words into sentences.
The complete message flow is shown in the following diagram:
The message flow in the example Storm-Kafka integration
We have already seen the WordsProducer class that produces words into the Kafka broker. Now, we will create a Storm topology that will read these words from Kafka and aggregate them into sentences. For this, we will have one KafkaSpout in the application that will read the messages from Kafka and two bolts: SentenceBolt, which receives words from KafkaSpout and then aggregates them into sentences which are then passed onto PrinterBolt, which simply prints them on the output stream. We will be running this topology in a local mode. Perform the following steps to create the Storm topology:
Create a new Maven project with the
com.learningstormgroup ID and thekafka-storm-topologyartifact ID.Add the following dependencies...