Transformation Patterns, Cleansing, and Normalization

In this chapter, we’ll learn about transformation patterns and their role in data management. The Lambda, Kappa, and Microservice architectural patterns will be covered in the following sections. We’ll also cover important data transformation methods, such as cleansing, normalization, masking, de-duplication, enrichment, validation, and standardization.

Data workers, like you, must understand these transformation patterns and methods. In a data-driven world, the ability to analyze raw data is invaluable. This expertise is crucial for data scientists preparing data for machine learning models, analysts gaining insights, and database administrators assuring data governance and security.

The Lambda, Kappa, and Microservice designs enable you to construct robust data pipelines for large and diversified data sources. Understanding data infrastructure construction is crucial in a business setting where fast and accurate...