Ingesting streaming data
An increasingly common source of data for analytic projects is data that is continually generated and needs to be ingested in near-real-time. Some common sources of this type of data are as follows:
- Data from IoT devices (such as smartwatches, smart appliances, and so on)
- Telemetry data from various types of vehicles (cars, airplanes, and so on)
- Sensor data (from manufacturing machines, weather stations, and so on)
- Live gameplay data from mobile games
- Mentions of the company brand on various social media platforms
For example, Boeing, the aircraft manufacturer, has a system called Airplane Health Management (AHM) that collects in-flight airplane data and relays it in real time to Boeing systems. Boeing processes the information and makes it immediately available to airline maintenance staff via a web portal.
In this section, we will look at several tools and services for ingesting streaming data, as well as things to consider...
