Hands-on – architecting a sample pipeline

For the hands-on portion of this chapter, you will review the detailed notes from a whiteboarding session held for the fictional company GP Widgets Inc. As you go through the notes, you should create a whiteboard architecture, either on an actual whiteboard or on a piece of poster board. Alternatively, you can create the whiteboard using a free online design tool, such as the one available at http://diagrams.net.

As a starting point for your whiteboarding session, you can use the following template. You can recreate this on your whiteboard or poster board, or you can access the diagrams.net/Draw.IO template for this via the GitHub site of this book at https://github.com/PacktPublishing/Data-Engineering-with-AWS-2nd-edition/blob/main/Chapter05/Data-Engineering-Whiteboard-Template.drawio.

Figure 5.7: Generic whiteboarding template

Note that the three zones included in the template (landing zone, clean zone, and curated...