Can't configure Grafana alerts with Loki

I have troubles configuring Grafana alerts with Loki. When I switch to the alerts page I always get the following:

In the browser devtools I see:
image

Here is the ruler part of my loki.yaml:

ruler:
  storage:
    type: local
    local:
      directory: /data/loki/rules
  rule_path: /data/loki/scratch
  alertmanager_url: http://alertmanager-operated.svc.cluster.local:9093
  ring:
    kvstore:
      store: inmemory
  enable_api: true
  enable_alertmanager_v2: true

What am I doing wrong?

In order to enable API for ruler the rules need to be stored in S3, and you have to interact with the ruler via API or cortextool (meaning you can’t just place the rules file at the local directory).

See Alerting and recording rules | Grafana Loki documentation about cortextool.

This is what I tried first, but I was unable to make it work.
Can you show me, how you would configure this?

This is my ruler configuration:

ruler:
  alertmanager_client:
    basic_auth_username: <AM_USER>
    basic_auth_password: <AM_PASS>
  alertmanager_url: <AM_URL>
  enable_api: true
  enable_sharding: true
  evaluation_interval: 1m
  ring:
    kvstore:
      store: memberlist
  rule_path: /tmp/loki/rules-temp
  storage:
    type: s3
    s3:
      bucketnames: <RULES_BUCKET>
      region: <REGION
      sse_encryption: true
      s3forcepathstyle: true
  evaluation:
    mode: remote
    query_frontend:
      address: <QF_ADDRESS>

The S3 bucket used for storing rules should be a different S3 bucket from what you use for Loki chunks and index storage.