Integration of source stream to Flink
Multiple sources are available to integrate with, the following is a list:
- Apache Kafka
- Amazon Kinesis Streams
- RabbitMQ
- Apache NiFi
- Twitter Streaming API
We will now see a demonstration of integration with Apache Kafka and RabbitMQ.
Integration with Apache Kafka
We have discussed Apache Kafka setup in previous chapters, so we will focus on Java code to integrate Flink and Kafka.
Follow the given steps:
- Add dependency in
pom.xml
:
<dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-java_2.11</artifactId> <version>1.2.0</version> </dependency>
The previous dependency is required for all type of the integration. The following dependencies are specific to Flink and Kafka integration:
<dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-connector-kafka-0.8_2.11</artifactId> <version>1.2.0</version> </dependency>...