Making it deployable
With the workflow thoroughly tested and established within your Jupyter Notebook, the next step is to enhance the code’s modularity. You can achieve this by organizing the code into separate scripts for extraction, transformation, and loading activities. By adopting a structured approach, you can ensure better maintainability and scalability of the ETL pipeline.
Design the following directory structure to create your deployable ETL pipeline:
project├── data │ ├── traffic_crashes.csv │ ├── traffic_crash_vehicle.csv │ └── traffic_crash_people.csv ├── etl │ ├── __init__.py │ ├── extract.py │ ├── transform.py │ └─...
