Hello,
I’m having an issue with Google Chat alterts. It used to work fine, I have created an template and I was recieving the notification on Google Chat with Title and the the details from the labels.
For the past few 2 months (I don’t know to be honest if it’s something from Percona Monitoring, but it’s Grafana based) I only recieve the Title of the notification (Not the labels - summary as I used to recieve). This issue only happens with Google Chat, on Telegram or email which I tried, I recieve the whole notification as it should be (Title and Text)
It used to be like this:
[FIRING:1] pmm_agent_down Alerting Rule (/agent_id/437486c0-e000-4520-81f0-230621d90ac4 pmm-agent 0 OS pmm-server pmm-managed /node_id/c1886df7-fe9e-4860-94db-d8673f9490e1 node_name 1 critical pmm_agent_down 2.41.2)
[FIRING:1] pmm_agent_down Alerting Rule (/agent_id/437486c0-e000-4520-81f0-230621d90ac4 pmm-agent 0 OS pmm-server pmm-managed /node_id/c1886df7-fe9e-4860-94db-d8673f9490e1 Node_name 1 critical pmm_agent_down 2.41.2)**Firing**
Value: [ var='A' labels={agent_id=/agent_id/437486c0-e000-4520-81f0-230621d90ac4, agent_type=pmm-agent, disabled=0, instance=pmm-server, job=pmm-managed, node_id=/node_id/c1886df7-fe9e-4860-94db-d8673f9490e1, node_name=node_name, version=2.41.2} value=1 ]
Labels:
- alertname = pmm_agent_down Alerting Rule
- agent_id = /agent_id/437486c0-e000-4520-81f0-230621d90ac4
- agent_type = pmm-agent
- disabled = 0
- grafana_folder = OS
- instance = pmm-server
- job = pmm-managed
- node_id = /node_id/c1886df7-fe9e-4860-94db-d8673f9490e1
- node_name = node_name
- percona_alerting = 1
- severity = critical
- template_name = pmm_agent_down
- version = 2.41.2
Annotations:
- description = PMM agent on node 'node_name', node ID '/node_id/c1886df7-fe9e-4860-94db-d8673f9490e1', cannot be reached. Host may be down.
- summary = PMM agent on node node_name' cannot be reached. Host may be down.```
Also tried another simple grafana 11.1.3 version selfhosted, has the same issue (Default settings on everything).
Grafana Cloud worked.