Further reading
This chapter covered different methods of ingesting data into your Bronze layer. Take a look at these resources to read more about the areas that interest you most:
- Use liquid clustering for Delta tables: https://docs.databricks.com/en/delta/clustering.html
- Spark Structured Streaming: https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html
- Delta Live Tables: https://docs.databricks.com/en/delta-live-tables/index.html
- DLT Databricks Demo: https://www.databricks.com/resources/demos/tutorials/lakehouse-platform/full-delta-live-table-pipeline
- Auto Loader options: https://docs.databricks.com/ingestion/auto-loader/options.html
- Schema evolution with Auto Loader: https://docs.databricks.com/ingestion/auto-loader/schema.html#configure-schema-inference-and-evolution-in-auto-loader
- Common loading patterns with Auto Loader: https://docs.databricks.com/ingestion/auto-loader/patterns.html
- Stream processing with Apache Kafka...
