We have a multi-cluster setup where we use Thanos in the observability cluster to query metrics from all other clusters.
Since we started using this setup we only managed rules locally and everything was working fine. I created my first PrometheusRule in my staging cluster, then checked it out in Thanos where I saw its in “firing” state, however Alertmanager doesn’t pick it up.
I was thinking that if that is not possible, maybe I can utilize Grafana alerting to handle Prometheus managed rules.
I saw that Grafana can only send alerts for rules that were created in Grafana, but I was wondering if still there’s another option that I don’t know about.