Mute alert on specific event

Grafana Alerting
1

Hi there,

I have a Grafana setup with prometheus, cadvisor, blackbox and node exporter.
It is monitoring a bunch of docker containers which are deployed with terraform.

The thing is :
When we remove a container which is not necessary anymore, we always receive alerts from grafana as all the metrics we are monitoring (such as http response) stops working, which is what is expected as the service was removed.

How would you address such case to avoid having alerts triggered for a service we just removed ?

Thanks

2

How are your alerts configured, can you share an example alert ?

3

Hi,

I’m using this request:
A: max(probe_success{}) by (instance)
B: Mean (avg) of A
C (alert condition): $B == 0
Range is now-10min