Grafana loki MS Teams (Grafana Version 10.4.1)

Sorry for delay. The template mentions {{CommonLabels.log_timestamp}} which may not be true if that label changes often and it’s not mentioned in group.

When the Alertmanager receives alerts (the one that you see in preview) that are identified by the set of labels, it processes them through notification policies, and then applied group_by parameter of the matching policy. The set of labels specified in group_by identify a group of alerts. That group lives in memory, and during the runtime Alertmanager can add and remove alerts from that group.
Now, every alert group is processed by a pipeline that is run every group_interval of the matching notification policy. When pipeline is run, it creates a snapshot of the group, which can contain one or many alerts. Then it calculates CommonLabels, i.e. labels have the same value for all alerts in the group.
When rule is evaluated every alert instance is uniquely identified by set of labels. Therefore, two alerts that have different values of the same label - different alerts.

Now back to the log_timestamp (and timestamp). That label seems to change every time the rule is evaluated, and therefore every time a new alert is created. However, due to group_by policy, all those alerts end up in the same group, which can cause that log_timestamp gets omitted from CommonLables.

I would recommend avoid such labels as much at possible because it can cause a lot of flapping alerts and quite useless state history.

To debug your alerts, I would recommend adding webhook integration that points to some debugging service such as webhook.site. So you can see exactly what’s going on with your alerts.