Processing event streams
At first glance, stream processing might seem like it is some extreme paradigm shift. And truth be told, if you have to do it all from scratch, it is not for the fainthearted. For example, we need a process that consumes from a stream and invokes the business logic. And we need one of these for each shard. And we need logic to increase or decrease the number of these processes with the number of shards. And we need to restart these processes when they fall over. And we need to keep track of the offset so that we read from the right position on the next call.
This can add up to a decent cloud bill, not to mention all the development elbow grease required. Of course, we can implement this processing as a container sidecar. But serverless, function-as-a-service offerings, such as AWS Lambda, provide this capability for just the cost of invoking the functions to execute the business logic. It takes care of all the heavy lifting.
Still, stream processing...