Summary

The number one goal of the streaming paradigm is to cater to low-latency applications. The Storm-on-YARN project has brought this paradigm to Hadoop. Stakeholders can now multiplex streaming and batch processing on a single Hadoop cluster and cater to different kinds of applications.

There are a number of streaming frameworks available, such as Microsoft SQL Server StreamInsight, S4, and Apache Storm, among others. Apache Storm is open source, part of the Apache Software Foundation, Hadoop integrated, and has a large community behind it, making it attractive for distributed stream processing.

Some key takeaways from this chapter are as follows:

• The basic data model in Apache Storm is an unbounded sequence of tuples called Streams.

• Long-standing queries are modeled as computational topologies. The data stream flows through these topologies.

• Apache Storm provides the following primitives:

  • Spouts: They convert input data into streams

  • Bolts: They take an input stream, do some processing on...