Cleansing and Transforming Data

Data engineers and analysts can spend massive amounts of time transforming, cleaning, and preparing data. They know it is not possible to generate accurate reporting and models with corrupted or incomplete data. With the variability of systems reports, you will likely encounter datasets that need to be manipulated in your future as a data analyst. In this chapter, we will dive deeper into data preparation. We will focus on cleansing and transforming data and provide you with approaches, strategies, and repeatable code and guidance that will help you improve the quality of your data in BigQuery.

Cleansing and transforming data can be done at various times in the data life cycle. Also known as data pre-processing, the goal of cleansing and transforming data is to enhance the performance of your data. We touched on transforming data alongside the loading process in Chapter 4, Loading and Transforming Data. Now, we will continue and discuss cleansing...