Understanding the design patterns for ETL

Even though the term was coined in 1958, business intelligence (BI) is a rather new discipline with a lot of research activity; its popularity in information systems has only gained popularity in the last few years (Reference #1). This rapid progress has also brought a high level of heterogeneity of several approaches for effective design patterns for data movement processes and design pattern norms within specific institutions (Reference #2). Thus, the formation of standardized, reliable, roadmaps for BI progress has slowly taken shape over time. The Extract, Transform, and Load (ETL) perspective for data pipelines has helped shaped the complexity of BI activities into neat building blocks to identify and represent frequently recurring patterns within an organization’s scope of data movement needs.

We chose to write this book from the plug and play perspective, which is why we chose to focus on the most common and versatile ETL design...