Analyzing Twitter feed

The example we are going to build will show how an application similar to the ad-tech use case can be built using the Apex library. Instead of processing a stream of ad-impression events from Kafka clusters, we will use a stream of tweets retrieved via the Twitter developer API. Instead of a dimensional model, we will compute windowed aggregates for selected metrics. The data visualization will take advantage of Grafana with a Pub/Sub plugin instead of a custom portal and frontend server. The goal will be to introduce the relevant building blocks and enable the reader to derive a similar application for their domain or use case.

The following sections will walk through the application functionality, its components, and a few selected implementation details. The full code, ready to run is available at the following link: https://github.com/tweise/apex-samples/tree/master/twitter.

The following is the DAG of the example application:

The input operator reads tweets from...