Running Prometheus on Kubernetes

Up until now, we have configured all scraping targets for Prometheus manually by adding them to the prometheus.yml configuration file. This works well for testing, but becomes tedious quickly in larger production setups (and completely pointless as soon as you introduce feature such as autoscaling).

When running your application within a Kubernetes cluster, Prometheus offers a turn-key solution for this—using the prometheus.yml configuration file, you can actually configure Prometheus to automatically load its scraping targets from the Kubernetes API. For example, if you have a Deployment defined for your booking service, Prometheus can automatically find all Pods that are managed by this Deployment and scrape them all. If the Deployment is scaled up, the additional instances will be automatically added to Prometheus.

For the following examples, we will assume that you have either a Minikube VM running on your local machine or a Kubernetes cluster somewhere...