Let's remember the Doubloon project requirements for the stream processing app. The customer sees BTC price event happens in the customer's web browser and is dispatched to Kafka via an HTTP event collector. The second step is to enrich the messages with the geolocation information. Remember from the previous chapter that defective messages result in bad data, so they are filtered.
Putting it all together, the specification is to create a stream application that does the following:
- Reads individual messages from a Kafka topic called raw-messages
- Validates the message, sending any invalid message to a dedicated Kafka topic called invalid-messages
- Enriches the message with the geolocation information
- Writes the enriched messages in a Kafka topic called valid-messages
All this is detailed in the following diagram and is the second version of the stream processing application:
Figure 4.1: The processing application reads events from the raw-messages topic, validates...