Summary

All in all, the backbone of any AI or ML pipeline is handling data. This means that anything that makes it easier to access and manipulate data while communicating it between tools is extremely important to building modular and composable systems. By leveraging memory mapping, protocols such as DLPack and the Arrow C data/device interface, and the tensor canonical extension types, more and more libraries and frameworks continue to adopt and benefit from the Arrow format and its implementations.

At this point, we’re going to steer away from direct development with Arrow for a bit and instead focus on examples of Arrow being used out in the wild. The next chapter is called Powered by Apache Arrow, and we’re going to look at existing applications and use cases that are powered by Arrow and look at how they use it. I want to shed a spotlight on some innovative projects that use Arrow in creative ways. Maybe you’ll even be familiar with one or more of the...