Modern Data Processing Architecture
Data architecture has become one of the most discussed topics. This chapter will introduce data architecture and the methodologies for designing a data ecosystem. Architecting a data solution is tricky and often riddled with traps. We will go through the theories for creating a data ecosystem and give some insight into how and why you would apply those theories.
To do so, we will cover the essential concepts, why they are helpful, and when to apply them.
By the end of this chapter, you will have built the foundation of your data solution, and once completed, you should be comfortable with architecture data solutions at a high level.
In this chapter, we’re going to cover the following main topics:
- Databases, data warehouses, and data lakes
- Data platform architecture at a high level
- Lambda versus Kappa architecture
- Lakehouse and Delta architectures
- Data mesh theory and practice
