Implementing quality controls throughout the data life cycle
Data quality should be a fundamental consideration throughout the entire life cycle of data. From data ingestion to utilization by downstream analytic teams, data undergoes various changes, and ensuring its quality at each step is paramount. Here is a diagram of the quality check life cycle:
Figure 2.1 – Quality check life cycle
Let’s understand more deeply what needs to happen at each step:
- Data entry/ingestion: Validating the data sources and ensuring that data is captured accurately and consistently while entering the system can limit errors in the downstream processes.
Data persona --> data engineer
- Data transformation: By incorporating quality checks into the data transformation layer, organizations ensure that data remains reliable, accurate, and consistent throughout its journey from raw sources to its final destination.
Data persona --> data engineer
...
