Spicing up your data workflows

Among the various fields of engineering that work with very large sets of data, one field that deals with processing some of the largest datasets would be ML and AI workflows. However, if your full-time job isn’t ML, and you don’t have the support of a dedicated ML team, it can often be very difficult to create an application that can learn and adapt. This is where a group of engineers decided to step in and make it easier for developers to create intelligent and adapting applications. Spice AI (https://spice.ai/) is, at the time of writing, a venture-capital-funded start-up that works to create a portable runtime for federating and co-locating data across multiple sources. They’ve open-sourced a product on GitHub called Spice.ai (https://github.com/spiceai/spiceai). It utilizes Apache Arrow and Arrow Flight SQL for communication and internal representations, which allow it to easily interact with systems such as DuckDB and Apache...