Streaming Arrow data between Python and Go
The C streaming API is a higher-level abstraction built on the initial ArrowSchema
and ArrowArray
structures to make it easier to stream data within a process across API boundaries. The design of the stream is to expose a chunk-pulling API that pulls blocks of data from the source one at a time, all with the same schema. The structure is defined as follows:
struct ArrowArrayStream { // callbacks for stream functionality int (*get_schema)(struct ArrowArrayStream*, struct ArrowSchema*); int (*get_next)(struct ArrowArrayStream*, struct ArrowArray*); const char* (*get_last_error)(struct ArrowArrayStream*); // Release callback and private data void (*release)(struct ArrowArrayStream*); &...