Extensibility is one of the important design principle followed rigorously by Kafka. The Kafka Connect tool makes Kafka extensible. The tool enables Kafka to connect with external systems and helps bring data into it, and also out from it to other systems. It has a common framework, using which custom connectors can be written. More details on Kafka connect can be found in Kafka documentation in https://kafka.apache.org/documentation.html#connect.
The following figure shows how Kafka Connect works.
Figure 13: Kafka connect working
The Kafka connectors are categorized into two:
- Source Connectors: Connectors which bring data into Kafka topics.
- Sink Connectors: Connectors which take data away from topics into other external systems
There are a huge list of connectors available, catering to various external systems, using which Kafka can hook onto them. These existing connectors are again categorized broadly into two:
- Certified connectors: Connectors which are written using the Kafka...