I’m using Grafana alerting with Loki as the data source.
In my LogQL query, I apply topk() over a metric extracted using unwrap.
Now I want to filter the result of topk to exclude any series where the label status = "start".
Is there a way to filter out those series after topk() — either in a Grafana alert rule or using a recording rule?
If so, how can I implement that?
My goal is to monitor the final state of containers, and only alert on those that have ended with a DIE or OOM status. So I’m using this query to extract and filter only the relevant failure cases.
my logql + metric query is like below
topk(1, last_over_time(
{container_name=~"docker-events-logger.*"}
| json
| Type = `container`
| status =~ `die|oom|start`
| unwrap timeNano [123s]
) by (Actor_Attributes_name, Actor_Attributes_image, host_name, host_ip, status)
) by (Actor_Attributes_name, Actor_Attributes_image, host_name, host_ip)