Overview of streaming

Stream processing is the act of continuously computing results as new data becomes available. A very simple example of this is computing the average of some numbers in a continuous fashion. To begin with, we start with the following information:

• Number of items = 0
• Current average = 0

As a new number comes in, we perform the following steps:

1. Compute a new total = Number of items x Current average + New number
2. Increment the number of items by one
3. Set the current average = New total / Number of items

As you can see, the continuous average computation algorithm is quite different from the batch-oriented algorithm. It is important to bear in mind the following facts when using this algorithm:

• The average value gets updated as new numbers become available
• The previously computed average value is reused to compute a new average

The following recipe using...