Loading Transformed Data
After data has undergone processing and transformation within an ETL pipeline, the final step involves transferring it to its designated final location. The type of output location that’s used is determined by both the data’s specific utility and the tools available within your environment.
It is most common to store output data iterations within structured, relational databases. Such databases offer an easily accessible format conducive to both analytical exploration and forthcoming modifications.
In this chapter, we’ll get more acquainted with the “L” of ETL pipelines and the best practices for designing efficient load operations. We will discuss how to optimize load activities to fit the output data using either incremental or full data loading activities. Finally, we’ll provide a comprehensive walkthrough for creating a relational database management system (RDBMS) on your local device that you can utilize...
