With Spark version 2.0 we have structured streaming which states that the output of the application is equal to executing a batch job on a prefix of the data. Structured Streaming handles consistency and reliability within the engine and in interactions with external systems. Structured Stream is a simple data frame and dataset API.
Users provide the query they want to run along with the input and output locations. The system then executes the query incrementally, maintaining enough state to recover from failure, keeping the results consistent in external storage, and so on.
Structured Streaming promises a much simpler model for building real-time applications, built on the features that work best in Spark Streaming. However Structured Streaming is in alpha for Spark 2.0.
