Multiple instances create one alert only

[mikkeschiren]

I have this alert using kube metrics in prometheus:

(sum_over_time(kube_pod_status_ready{namespace=~"prod|staging", condition="true", pod=~"matomo-.+"}[1d:1s]) > 21600) and (kube_pod_status_ready{namespace=~"prod|staging", condition="true", pod=~"matomo-.+"}==1)

It is looking for cron jobs with more than six hours execution time. If one cron job exceeds that I get the alert, but if another one also does (there could be many cron jobs running at the same time), I do not get an alert for the new one, is it possible to solve this without creating different alerts for the different cron jobs?

[mattabrams]

Hi @mikkeschiren,

Are you still experiencing this issue? Have you tried adjusting your alert to limit it to one namespace? I’m just thinking of ways that you could remove complexity to troubleshoot the underlying issue…