One of the important characteristics of any streaming solution is that it serves as an integration platform as well. It collects events from varied sources and performs processing on these different events to produce the desired outcomes. One of the pertinent problems with such integration platforms is different data formats. Each type of source has its own format. Some support XML formats and some support JSON or Avro formats. It is difficult for you to design a solution catering to all formats. Moreover, as more and more data sources get added, you need to add support for data formats supported by the newly added source. This is obviously a maintenance nightmare and buggy.
Ideally, your streaming solution should support one data format. Events should be in the key/value model. The data format for these key/value events should be one agreed-on format. You should...
