Swimming in data with Dremio Sonar
The roots of Arrow can be found in the ValueVector objects from the Apache Drill project, a SQL query engine for Hadoop, NoSQL, and cloud storage. Dremio Sonar was originally built out of Apache Drill and one of Dremio’s founders co-created Arrow. Arrow is used by Dremio Sonar as the internal memory representation for its query and calculation engine, which helps power its performance. Since its inception, Dremio’s engineers have made many contributions to the Arrow project, resulting in significant innovations. First, let’s look at the architecture used and where Arrow fits in.
Clarifying Dremio Sonar’s architecture
As a distributed query engine, Dremio Sonar can be deployed in many different environments and scenarios. However, at its core, it has a pretty simple architecture, as shown in Figure 10.1. Being distributed, it can scale horizontally by increasing the number of coordinators and executors, which handle...
