Loki errors when viewing some alert rules history

Grafana Alerting
1

OS: Debian Trixie 13 on a Raspberry PI 5
Grafana: 13.1.1
Loki: 3.7.2 (branch: release-3.7.x, revision: 7486c4a7)

Loki configuration changes for alert state per documentation.
limits_config:
split_queries_by_interval: ‘24h’
max_query_parallelism: 32

Grafana configuration changes for alert state.
[unified_alerting.state_history]
enabled = true
backend = loki
loki_remote_url = http://localhost:3100

[feature_toggles]
#enable = alertingCentralAlertHistory - change to newer method.
alertingCentralAlertHistory = true

I can look at the main history page and not get any loki errors.

Screenshot 2026-07-22 105356
Screenshot 2026-07-22 1053561732×1008 171 KB

I can look at some alert rules with no loki errors.
NO ERRORS:

Screenshot 2026-07-22 105113
Screenshot 2026-07-22 1051131732×1008 186 KB

This one has loki ERRORS:

Screenshot 2026-07-22 105150
Screenshot 2026-07-22 1051501732×1008 191 KB

Loki errors:

1784727654113 2026-07-22T13:40:54.113Z <30>1 2026-07-22T09:40:54.113934-04:00 raspberry05 loki 230844 - - level=error ts=2026-07-22T13:40:54.055233402Z caller=scheduler_processor.go:176 component=querier org_id=fake msg=error notifying scheduler about finished query err=EOF addr=127.0.0.1:9096
1784727654014 2026-07-22T13:40:54.014Z <30>1 2026-07-22T09:40:54.014076-04:00 raspberry05 loki 230844 - - level=error ts=2026-07-22T13:40:54.006316047Z caller=scheduler_processor.go:176 component=querier org_id=fake msg=error notifying scheduler about finished query err=EOF addr=127.0.0.1:9096
1784727654014 2026-07-22T13:40:54.014Z <30>1 2026-07-22T09:40:54.014051-04:00 raspberry05 loki 230844 - - level=error ts=2026-07-22T13:40:54.005943694Z caller=scheduler_processor.go:176 component=querier org_id=fake msg=error notifying scheduler about finished query err=EOF addr=127.0.0.1:9096
1784727654014 2026-07-22T13:40:54.014Z <30>1 2026-07-22T09:40:54.014032-04:00 raspberry05 loki 230844 - - level=error ts=2026-07-22T13:40:54.004544338Z caller=scheduler_processor.go:111 component=querier msg=error processing requests from scheduler err=rpc error: code = Canceled desc = context canceled addr=127.0.0.1:9096
1784727654014 2026-07-22T13:40:54.014Z <30>1 2026-07-22T09:40:54.014013-04:00 raspberry05 loki 230844 - - level=error ts=2026-07-22T13:40:54.004505153Z caller=scheduler_processor.go:111 component=querier msg=error processing requests from scheduler err=rpc error: code = Canceled desc = context canceled addr=127.0.0.1:9096
1784727654013 2026-07-22T13:40:54.013Z <30>1 2026-07-22T09:40:54.013993-04:00 raspberry05 loki 230844 - - level=error ts=2026-07-22T13:40:54.004408782Z caller=scheduler_processor.go:111 component=querier msg=error processing requests from scheduler err=rpc error: code = Canceled desc = context canceled addr=127.0.0.1:9096
1784727654013 2026-07-22T13:40:54.013Z <30>1 2026-07-22T09:40:54.013969-04:00 raspberry05 loki 230844 - - level=error ts=2026-07-22T13:40:54.003599095Z caller=scheduler_processor.go:176 component=querier org_id=fake msg=error notifying scheduler about finished query err=EOF addr=127.0.0.1:9096
1784727654013 2026-07-22T13:40:54.013Z <30>1 2026-07-22T09:40:54.013886-04:00 raspberry05 loki 230844 - - level=error ts=2026-07-22T13:40:54.003256371Z caller=scheduler_processor.go:111 component=querier msg=error processing requests from scheduler err=rpc error: code = Canceled desc = context canceled addr=127.0.0.1:9096
1784727644913 2026-07-22T13:40:44.913Z <30>1 2026-07-22T09:40:44.91362-04:00 raspberry05 loki 230844 - - level=error ts=2026-07-22T13:40:44.840679261Z caller=scheduler_processor.go:176 component=querier org_id=fake msg=error notifying scheduler about finished query err=EOF addr=127.0.0.1:9096
1784727644814 2026-07-22T13:40:44.814Z <30>1 2026-07-22T09:40:44.814805-04:00 raspberry05 loki 230844 - - level=error ts=2026-07-22T13:40:44.788957378Z caller=scheduler_processor.go:176 component=querier org_id=fake msg=error notifying scheduler about finished query err=EOF addr=127.0.0.1:9096
1784727644814 2026-07-22T13:40:44.814Z <30>1 2026-07-22T09:40:44.81479-04:00 raspberry05 loki 230844 - - level=error ts=2026-07-22T13:40:44.764882582Z caller=scheduler_processor.go:176 component=querier org_id=fake msg=error notifying scheduler about finished query err=EOF addr=127.0.0.1:9096
1784727644814 2026-07-22T13:40:44.814Z <30>1 2026-07-22T09:40:44.814774-04:00 raspberry05 loki 230844 - - level=error ts=2026-07-22T13:40:44.758363154Z caller=scheduler_processor.go:176 component=querier org_id=fake msg=error notifying scheduler about finished query err=EOF addr=127.0.0.1:9096
1784727644814 2026-07-22T13:40:44.814Z <30>1 2026-07-22T09:40:44.814757-04:00 raspberry05 loki 230844 - - level=error ts=2026-07-22T13:40:44.758140524Z caller=scheduler_processor.go:111 component=querier msg=error processing requests from scheduler err=rpc error: code = Canceled desc = context canceled addr=127.0.0.1:9096
1784727644814 2026-07-22T13:40:44.814Z <30>1 2026-07-22T09:40:44.814732-04:00 raspberry05 loki 230844 - - level=error ts=2026-07-22T13:40:44.757909116Z caller=scheduler_processor.go:111 component=querier msg=error processing requests from scheduler err=rpc error: code = Canceled desc = context canceled addr=127.0.0.1:9096
1784727644814 2026-07-22T13:40:44.814Z <30>1 2026-07-22T09:40:44.814711-04:00 raspberry05 loki 230844 - - level=error ts=2026-07-22T13:40:44.75788606Z caller=scheduler_processor.go:111 component=querier msg=error processing requests from scheduler err=rpc error: code = Canceled desc = context canceled addr=127.0.0.1:9096
1784727644814 2026-07-22T13:40:44.814Z <30>1 2026-07-22T09:40:44.814611-04:00 raspberry05 loki 230844 - - level=error ts=2026-07-22T13:40:44.757839504Z caller=scheduler_processor.go:111 component=querier msg=error processing requests from scheduler err=rpc error: code = Canceled desc = context canceled addr=127.0.0.1:9096

Since these seem to be a Grafana generated issue, is there maybe a configuration that can be added or adjusted?

2

Since the main Alert History page works, and the History tab also works for another alert rule, this doesn’t appear to be a general alert state history configuration issue.

The issue appears to be specific to this alert rule. I’d first try reducing the History time range (for example, Last 24 hours) and see whether the scheduler_processor.go context canceled/EOF messages still occur. If the behavior changes with a smaller time range, that would suggest the amount of data being queried is a factor.

If the issue is consistently reproducible only for this rule, while other alert history pages continue to work, I’d recommend opening a Grafana or Loki GitHub issue with the rule definition, your Grafana and Loki versions, Loki configuration, steps to reproduce, and the corresponding Loki logs.

3

How would you reduce the history time range for an alert rule?

6

What settings and where please?

10

Do you have any specific information on where to change these settings?

14

I don’t know what UI you are asking me to use to get there.

16

What UI?

18

What would looking at settings on a phone do to solve my Grafana/Loki issue?

22

Quick update.
I reduced the Grafana setting loki_max_query_length to 90h from 721h. I have not seen any of the loki errors since. Seems there should be some Loki tuning opportunities that may allow me to increase the Grafana setting back at least some.