Kafka on Mesos
This section will introduce Kafka and explain how to set it up on Mesos while also discussing the problems commonly encountered during the setup process.
Introduction to Kafka
Kafka is a distributed publish-subscribe messaging system designed for speed, scalability, reliability, and durability. Some of the key terms used in Kafka are given as follows:
Topics: These are the categories where message feeds are maintained by Kafka
Producers: These are the upstream processes that send messages to a particular Kafka topic
Consumers: These are the downstream processes that listen to the incoming messages in a topic and process them as per requirements
Broker: Each node in a Kafka cluster is called a broker
Take a look at the following high-level diagram of Kafka (source: http://kafka.apache.org/documentation.html#introduction):
A partitioned log is maintained by the Kafka cluster for every topic, which looks similar to the following (source: http://kafka.apache.org/documentation.html...
