Loki alerting via ruler doesn't work

Grafana Loki
,
1

Ubuntu 20.04.4 LTS; Grafana v7.5.15

Ruler block in config:

ruler:
  storage:
    type: local
    local:
      directory: /etc/loki/rules
  rule_path: /tmp/loki/rules-temp
  alertmanager_url: http://localhost:9093
  ring:
    kvstore:
      store: inmemory
  enable_api: true
  enable_alertmanager_v2: true
storage_config:
  boltdb:
    directory: /var/lib/loki/index
  filesystem:
    directory: /var/lib/loki/chunks

test.rules in /etc/loki/rules

- name: test_alerts
  rules:
    - alert: test_alert
      annotations:
        message: "{{ $labels.pod }} is alerting by test rule"
      expr: 'count_over_time({pod=~".+"}[1m] |~ ".+") > 200'
      for: 0m
      labels:
        severity: critical

The rules I expect to look when making HTTP request but don’t get:

curl -sX GET http://127.0.0.1:3100/prometheus/api/v1/rules | jq .
{
  "status": "success",
  "data": {
    "groups": []
  },
  "errorType": "",
  "error": ""
}

and no one alert in alertmanager

2

The way the issue was resolved is to move all rules from /etc/loki/rules to /etc/loki/rules/fake
The reason why it should work such way is not clear for me.

1 Like
3

The reason for this is because the rules need to be storied under your tenant id, so where you specified fake, it should actually be your tenant id.