Data Discovery – Understanding Our Data before Ingesting It
As you may already have noticed, data ingestion is not just retrieving data from a source and inserting it in another place. It involves understanding some business concepts, secure access to the data, and how to store it, and now it is essential to discover our data.
Data discovery is the process of understanding our data’s patterns and behaviors, ensuring the whole data pipeline will be successful. In this process, we will understand how our data is modeled and used, so we can set up and plan our ingestion using the best fit.
In this chapter, you will learn about the following:
- Documenting the data discovery process
- Configuring OpenMetadata
- Connecting OpenMetadata to our database
