Going through the foundational concepts for data engineering
Even though there are many data engineering concepts that we will learn throughout the book by using Google Cloud Platform (GCP), there are some basic concepts that you need to know as data engineers. In my experience of interviewing in data companies, I discovered that these foundational concepts are often asked to assess how much you know about data engineering. Take the following examples:
- What is ETL?
- What’s the difference between ETL and Extract, Load, and Transform (ELT)?
- What is big data?
- How do you handle large volumes of data?
These questions are quite common, yet particularly important to deeply understand the concepts since they may affect our decisions on architecting our data life cycles.
ETL concept in data engineering
ETL is the key foundation of data engineering. Everything in the data life cycle is ETL; any part that happens from upstream to downstream is ETL. Let&...
