Managing Metrics in Kubernetes

Once upon a time, monitoring and metrics were a complex and very proprietary corner of the industry. While there were some open-source projects that did monitoring, the majority of “enterprise” systems were large, cumbersome, and proprietary. There were a few standards, such as SNMP, but for the most part, every vendor had their own agents, their own configurations, their own…everything. If you wanted to write an application that generated metrics or alerts, then you needed to write to their SDK. This led to monitoring being one of the centralized services, like databases, but required much deeper understanding of what’s being monitored. Changes were difficult and ultimately, many systems followed either you only live once (YOLO) monitoring or very basic high-level monitoring that “checked the compliance box,” but didn’t provide much value.

Then came the Prometheus project, which made two critical...