Chapter 5: Data Consolidation in Delta Lake
In the previous chapters, we discussed the quality of Delta and why it has become the first choice in big data processing. In this chapter, we will focus on how to consolidate disparate datasets into one or more data lakes backed by Delta so that you can build all kinds of use cases on a single source of truth without having to move data or stitch together multiple systems. We have already looked into the special features that Delta offers, including ACID transaction support, schema evolution, time travel, fine-grained data operations, and also big data design blueprints (such as the medallion architecture) in the context of data workflows. In this chapter, we will use those...
