ETL is a process of Extracting, Transforming, and Loading data into the target system, which is explained next. It is followed by a large number of organizations to build their data pipelines.
-
Extraction: Extraction is the process of ingesting data from the source system and making it available for further processing. Any prebuilt tool can be used to extract data from the source system. For example, to extract server logs or Twitter data, you can use Apache Flume, or to extract data from the database, you can use any JDBC-based application, or you can build your own application. The objective of the application that will be used for extraction is that it should not affect the performance of the source system in any manner.
- Transformation: Transformation refers to processing extracted data and converting it into some meaningful...
