Pushing data from Kafka to Elastic
As mentioned, Elasticsearch is a distributed, full-text search engine that supports a RESTful web interface and schema-free JSON documents. Elasticsearch was built with distributed searches in mind. There are several ways to push data into Elasticsearch. In this recipe, the plugin that enables data pushing from Kafka to Elasticsearch is analyzed.
Getting ready
For this recipe, a Kafka cluster must be up and running and a confluent platform is needed. To install Elasticsearch follow the instructions on this page:Â https://www.elastic.co/guide/en/elasticsearch/reference/current/_installation.html.
How to do it...
The Elasticsearch connector is needed. Before starting the connector, the configuration is in
etc/kafka-connect-elasticsearch/quickstart-elasticsearch.properties.
This must be properly set to the Elasticsearch configuration: connection.url points to the correct HTTP address.
To start the Elasticsearch connector, use the following command:
$ ./bin/connect...
