Input sources and output stores
Spark Streaming supports three kinds of input sources:
Basic sources: Sources directly available in the StreamingContext API. Examples: file systems, socket connections, and Akka actors.
Advanced sources: Sources like Kafka, Flume, Kinesis, Twitter, and so on, which are available through extra utility classes.
Custom sources: Requires implementing a user-defined receiver.
Multiple receivers can be created in the same application to receive data from different sources. It is important to allocate enough resources (cores and memory) for enabling receivers and tasks to execute simultaneously. For example, if you start your application with one core, it will be taken by the receiver and no tasks will be executed because of a lack of available cores.
Basic sources
There are four basic sources available in Spark StreamingContext as shown in the following table:
|
Source |
Description |
|---|---|
|
TCP stream |
For streaming data via TCP/IP by specifying a hostname and a port number... |
