Hello,
I am using the Grafana Operator 4.2.0 on OpenShift 4.9. Grafana Operator 4.2.0 is the most recent and it deploys Grafana 7.5. So I cannot use Grafana 8.x because the Operator I use does not offer it.
So it means my Grafana instance (v7.5) is connected to the Prometheus instance provided in the OpenShift Monitoring stack. In fact two Prometheus instances are running on my OCP cluster. One for the system metrics, one for the user projects metrics. Both of them are connected linked together with Thanos Querier which is my DataSource in Grafana.
I can get metrics from user project Cert-Manager that is issuing certificates across the OpenShift cluster. I created a dashboard with useful information and I configured an alert when renewal time is close.
I configured the alert to send a MS Teams notification and it works fine. I used some templating but I have issues with it.
The ALERTING event shows the $name correctly but I cannot figure out to show the $value which would represent the time left when the alert is firing. Also at the bottom I get the query result with all the Labels and the Value but I do not know how to hide this. Here is my description string from the alert configuration tab:
Certificate ${name} from the namespace ${exported_namespace} will expire soon in ${value} seconds.
Here is what the query returns:
{container="cert-manager", endpoint="http", exported_namespace="jfrog2", instance="10.130.1.253:9402", job="cert-manager", name="jfrog-internal-tls", namespace="cert-manager", pod="cert-manager-679994f9b4-r4qpk", prometheus="openshift-user-workload-monitoring/user-workload"}
182.22699999809265
Here is what the alert looks like in Teams:
Also, when I get the OK notification, the $name is not replaced anymore. Is it normal templating does not work for resolved events?
Here is how the alert name is configured
Certificate ${name} expires soon
To summarize here are the questions:
1- What is the notation to print the query value in my description?
2- How to remove the query result at the bottom of my Teams notification?
3- Why the OK notification is not resolving template for certificate $name?