Hi everyone, recently we just enabled the Loki mixin from upstream: loki/production/loki-mixin at main · grafana/loki · GitHub, and
LokiRequestLatency (loki/alerts.libsonnet at main · grafana/loki · GitHub) gets fired constantly since the p99 latency of querier and query frontend for my environment sometimes is around 4s.
My questions are mainly:
- How to solve this permanently? does scale up/out querier/query-frontend help?
- The severity of this alert is
critical, does it deserve it? or
warningshould be OK?