Example implementations

Now that we see the value in the pipeline pattern, let's start planning a Go implementation of one.

In Go, pipelines are implemented using a series of stages connected by Go channels. A Go pipeline begins with a data source (aka producer), has stages that are connected via channels, and ends with a data sink (aka consumer).

The data source can be a generator function that sends data to the first stage and then closes the initial outbound channel.

Each filter (step or stage) in the pipeline:

• Consists of one or more Goroutines that run the same function (aka filter)
• Receives upstream data via one or more inbound channels
• Transforms the data in some way
• Sends data downstream via one or more outbound channels
• Closes its outbound channels when all the send operations are completed
• Keeps receiving values from inbound channels until those channels are closed

Example transformer functions include the following: 

• Accumulator
• Aggregator
• Delta (to calculate the change between two sample...