In the previous chapter, we learned how to ingest and transform data to train or evaluate a model using a batch ETL approach. You would use this approach in the training or evaluation phases in most cases, but when running a model, streaming ingestion is needed. This chapter covers setting up streaming ingestion strategies for DL models using a combination of the Apache Spark, DL4J, DataVec, and Apache Kafka frameworks. Streaming data ingestion frameworks don't simply move data from source to destination such as in the traditional ETL approach. With streaming ingestion, any incoming data in any format can be simultaneously ingested, transformed, and/or enriched with other structured and previously stored data for DL purposes.
In this chapter, we will cover the following topics:
- Streaming data with Apache Spark
- Streaming data with Kafka and Apache Spark
- Streaming...
