There are currently two worlds in Grafana: old alerts linked to panels that show a red/green heart for ease of visual inspection, and the new unified alerting that people are use more for proper monitoring/alerting.
These two worlds are managed in “Alerting > Alert Rules”. I think this is bad because people have to come up with clever naming schemes to differentiate between them.
For example, if I have a panel showing “ElasticSearch query time” and I want some visual aid to show things are broken, I’ll add an alert to that panel without any labels (so no notification policy triggers).
I will certainly have a similar named alert for proper monitoring, that will have labels and will send notifications. I cannot name both of them the same.
I would prefer if “panel alerts” weren’t considered part of unified alerting. If they only existed in the panel’s configuration and never showed in “Alerting > Alert Rules”.
I think this come from the fact that Grafana evolved two major feature sets (dashboards and alerts) in a single product. I have said it a few times, they are clashing.
I’d be fine with “de-evolving” panel alerts into just a mere threshold configuration that must use the same query from the panel itself. Not a separate query like today.