Running Kafka on Mesos

According to the definition, Apache Mesos is an open source project to manage computer clusters. This recipe shows how to run the Kafka on Mesos framework.

Getting ready

The following applications must be available on the machine:

• Java version 7 or later (http://openjdk.java.net/install/)
• Gradle (http://gradle.org/installation)

How to do it...

1. To download the Kafka on Mesos project from the repository, type the following command:

$ git clone https://github.com/mesos/kafka$ cd kafka $ ./gradlew jar 

2. The following command downloads the Kafka executor:

$ wget https://archive.apache.org/dist/kafka/0.11.0.0/kafka_2.11-0.11.0.0.tgz

3. Set this environment variable pointing to the libmesos.so file:

$ export MESOS_NATIVE_JAVA_LIBRARY=/usr/local/lib/libmesos.so 

4. Use the kafka-mesos.sh script to launch and configure Kafka on Mesos, but first create the kafka-mesos.properties file containing this:

storage=file:kafka-mesos.json  
master=zk://master:2181/mesos  
zk=master:2181  
api=http://master...