Unifying batch and real time using Lambda Architecture
Both batch and real-time data processing are important elements of any modern Enterprise DSS, and an architecture that seamlessly implements both these data processing techniques can help increase throughput, minimize latency, and allow you to get to fresh data much more quickly. One such architecture is called Lambda Architecture, which we will examine next.
Lambda Architecture
Lambda Architecture is a data processing technique that is used to ingest, process, and query both historical and real-time data with a single architecture. Here, the goal is to increase throughput, data freshness, and fault tolerance while maintaining a single view of both historical and real-time data for end users. The following diagram illustrates a typical Lambda Architecture:
Figure 2.3 – Lambda Architecture
As shown in the preceding diagram, a Lambda Architecture consists of three main components, namely, the...
