Apache Storm is a free and open source distributed real-time stream processing framework. At the time of writing this book, the stable release version of Apache Storm is 1.0.5. The Storm framework is predominantly written in the Clojure programming language. Originally, it was created and developed by Nathan Marz and the team at Backtype. The project was later acquired by Twitter.
During one of his talks on the Storm framework, Nathan Marz talked about stream processing applications using any framework, such as Storm. These applications involved queues and worker threads. Some of the data source threads write messages to queues and other threads pick up these messages and write to target data stores. The main drawback here is that source threads and targets threads do not match the data load of each other and this results in data pileup. It also results in data loss...
