Creating a data lake

One of the major benefits of the Event Sourcing pattern is that it results in an audit trail of all the state changes within a system. This store of events can also be leveraged to replay events to repair broken services and seed new components. A cloud-native event stream, such as AWS Kinesis, only stores events for a short period of time, ranging from 24 hours to 7 days. An event stream can be thought of as a temporary or temporal event store that is used for normal, near real-time operation. In the Creating a micro event store recipe, we will discuss how to create specialized event stores that are dedicated to a single service. In this recipe, we will create a data lake in S3. A data lake is a perpetual event store that collects and stores all events in their raw format in perpetuity with complete fidelity and high durability to support auditing and replay...