Exploring Apache Arrow Flight RPC

Distributed systems have always interested me. A distributed system is like a really good puzzle; it’s immensely satisfying once you figure out how all the pieces fit together to achieve your goal. If you’re not familiar with the term, a distributed system is simply a situation where you have various components of a system spread across multiple machines on a network. The idea is to split up the work and coordinate efforts among the components to complete tasks more efficiently. A great example would be Apache Spark.

The goal of distributed systems is generally to provide a robust, scalable, and reliable conglomeration of components that efficiently perform operations by distributing work across a system. This often means large amounts of data flowing between various components so that the data can get processed, manipulated, or otherwise operated on. When it comes to Apache Arrow-formatted data, the Arrow project provides a remote...