Building data pipelines with Kafka and NiFi
To build a data pipeline with Apache Kafka, you will need to create a producer since we do not have any production Kafka clusters to connect to. With the producer running, you can read the data like any other file or database.
The Kafka producer
The Kafka producer will take advantage of the production data pipeline from Chapter 11, Project — Building a Production Data Pipeline. The producer data pipeline will do little more than send the data to the Kafka topic. The following screenshot shows the completed producer data pipeline:
To create the data pipeline, perform the following steps: