In this recipe, we explore the concept of a streaming DataFrame. We create a DataFrame consisting of the name and age of individuals, which we will be streaming across a wire. A streaming DataFrame is a popular technique to use with Spark ML since we do not have a full integration between Spark structured ML at the time of writing.
We limit this recipe to only the extent of demonstrating a streaming DataFrame and leave it up to the reader to adapt this to their own custom ML pipelines. While streaming DataFrame is not available out of the box in Spark 2.1.0, it will be a natural evolution to see it in later versions of Spark.