Ingesting streaming data

Although you can ingest a variety of data into Data Explorer for analysis, for our purposes we are going to look at ingesting IoT data streams.

Streaming ingestion is valuable for data loading when there is a need for minimal delay between the process of ingesting data and querying it. It is advisable to employ streaming ingestion in the following situations:

• When a latency of less than one second is necessary
• To enhance the operational processing of numerous tables where the flow of data into each table is relatively small (a few records per second), while the overall data ingestion volume is substantial (thousands of records per second)
• If the flow of data into each table is substantial (over 4 GB per hour), it is recommended to use batch ingestion

There are two types of ingestion supported in ADX: data connection and custom ingestion:

• Data connection: You can use this if the data is coming from Event Hubs, IoT Hub, and Event...