Multiple sources are available to integrate with, the following is a list:
- Apache Kafka
- Amazon Kinesis Streams
- RabbitMQ
- Apache NiFi
- Twitter Streaming API
We will now see a demonstration of integration with Apache Kafka and RabbitMQ.
We have discussed Apache Kafka setup in previous chapters, so we will focus on Java code to integrate Flink and Kafka.
Follow the given steps:
- Add dependency in
pom.xml
:
<dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-java_2.11</artifactId> <version>1.2.0</version> </dependency>
The previous dependency is required for all type of the integration. The following dependencies are specific to Flink and Kafka integration:
<dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-connector-kafka-0.8_2.11</artifactId> <version>1.2.0</version> </dependency>...