Tackling common data challenges

Let’s look at some common considerations that are applicable across many of the ESG scenarios we covered throughout this chapter, such as idempotence, data enrichment, latching, and resynchronizing slow data.

Idempotence

Idempotence is an important piece of the event-first approach. We covered different approaches in Chapter 4, Trusting Facts and Eventual Consistency, and Chapter 5, Turning the Cloud into the Database. The external systems that ESGs integrate with may or may not support idempotence. For example, a third-party SaaS product most likely will support it, but a legacy system probably will not. The legacy system’s API will dictate which approach we can use, such as using Direct SQL to implement an inverse OpLock.

If the external system does not provide idempotence, then we can implement it in the ESG with a micro event store. Figure 7.21 depicts the resources involved:

Figure 7.21: Egress – idempotence...