Analyzing real-time data is demanded by all enterprises in this digital age where data is at its core. Elasticsearch can play a very important role in dealing with such real-time data along with other stream processors (in our case, it is Apache Flink). The following figure shows a typical setup used for such data handling and is quite relevant with regard to our technology choice and use case implementation. In place of Flink, any other stream processors could be used, say Spark Streaming, to achieve the architecture mentioned here:
Figure 18: Elasticsearch setup in real-time data handling in conjunction with Flink
This architecture is quite relevant and useful because Flink can do analysis and transformation of data and after that Elastic Stack can be used in the serving layer for fast queries on that data. The built-in component, namely Kibana, in Elastic Stack is an eye...
