I would like to create a dashboard with a table-like view that shows just the deployed versions of Kubernetes pod’s across multiple Kubernetes environments.
The table would have columns for:
- pod-name
- K8-env-1 (e.g. AU)
- K8-env-2 (e.g. US)
- K8-env-3 (e.g. UK)
- etc
There would be a row for each type of pod, e.g.
- account-service
- billing-service
- etc
The cell/intersection of the rows and columns would have just the version e.g. “v1.2.1”, “v3.2.1”.
My pod metrics seem to have a label called “image” which has (amongst other things) the version of the pod. So I can extract the version out of there.
From this dashboard, I would be able to quickly see/compare deployed software versions globally.
We already have K8, Prometheus, and Grafana deployed globally, and I know how to create a Datasource variable of type Prometheus which has access to all global/regional instance data.
Some problems that I face are:
- The metrics out of each region are almost indistinguishable, they don’t have a label on them that says where(which region) they came from. Perhaps we need to modify/configure the component that scrapes the pods in K8 and loads into Prometheus.
- I don’t know how best to build a dash with this info.
I got something working using the number of regions * the number of podtypes stat panels, created manually, but it is fiddly, slow to build, slow to update, and brittle. I want a dash that as much as possible discovers new pod types, and even new regions if possible.
We are using AWS Hosted Grafana and AWS EKS for K8.
I feel like my requirement could be a common one, so I am wondering if there is something already out there, or if anyone has any tips.