Evolution of data architectures
We will start with understanding how data architectures traditionally have been followed by detailing the demands of modern machine learning or analytics platforms in the context of big data.
Observation 1—Data stores were always for a purpose
Traditionally, data architectures had a clear segregation of purpose, OLTP (Online Transaction Processing), typically known to be used for transactional needs, and OLAP (Online Analytic Processing) data stores that typically used for reporting and analytical needs. The following table elaborates the general differences:
|
OLTP databases |
OLAP databases | |
|---|---|---|
|
Definition |
This involves many small online transactions (INSERT, UPDATE, and DELETE). The fast query processing is the core requirement; maintaining data integrity, concurrency, and effectiveness is measured by the number of transactions per second. It's usually characterized by a high-level of normalization. |
This involves a relatively small volume... |
