Streaming sources and sinks
Streaming sources and sinks are essential components in a streaming system that enable the ingestion of data from external systems and the output of processed data to external destinations. They form the connectors between the streaming application and the data sources or sinks.
Streaming sources retrieve data from various input systems, such as message queues, filesystems, databases, or external APIs, and make it available for processing in a streaming application. On the other hand, streaming sinks receive processed data from the application and write it to external storage, databases, filesystems, or other systems for further analysis or consumption.
There are different types of streaming sources and sinks. We will explore some of them next.
Built-in streaming sources
Structured Streaming provides built-in support for a variety of streaming sources, making it easy to integrate with popular data systems. Some of the commonly used built-in streaming...
