Data plane telemetry aggregation

After collecting and parsing data plane telemetry, we need to think about what to do with it next. Looking at raw data is not always helpful because of the sheer number of flows and lack of any meaningful context. Hence, the next logical step in a telemetry processing pipeline is data enrichment and aggregation.

Telemetry enrichment refers to the process of adding extra metadata to each flow based on some external source of information. For example, these external sources can provide a correlation between a public IP and its country of origin or BGP ASN, or between a private IP and its aggregate subnets or device identity.

Another technique that can help us interpret and reason about the telemetry we collect is aggregation. We can combine different flow records either based on the IP prefix boundary or flow metadata, such as a BGP ASN, to help network operators draw meaningful insights and create high-level views of the data.

You could build...