Exemplar query failed log spam

Grafana Dashboards
1

Recently my Grafana logs consist mainly from logs about Examplar query failed:

2022-12-05T14:02:43.104813094Z logger=tsdb.prometheus t=2022-12-05T15:02:43.104574728+01:00 level=error msg="Exemplar query failed" query=105 err="[]v1.ExemplarQueryResult: decode slice: expect [ or n, but found \u0000, error found in #0 byte of ...||..., bigger context ...||..."
2022-12-05T14:02:43.128637965Z logger=tsdb.prometheus t=2022-12-05T15:02:43.128418899+01:00 level=error msg="Exemplar query failed" query=4.80 err="[]v1.ExemplarQueryResult: decode slice: expect [ or n, but found \u0000, error found in #0 byte of ...||..., bigger context ...||..."
2022-12-05T14:02:48.216753949Z logger=tsdb.prometheus t=2022-12-05T15:02:48.216579054+01:00 level=error msg="Exemplar query failed" query=105 err="[]v1.ExemplarQueryResult: decode slice: expect [ or n, but found \u0000, error found in #0 byte of ...||..., bigger context ...||..."
2022-12-05T14:02:48.284472849Z logger=tsdb.prometheus t=2022-12-05T15:02:48.283612031+01:00 level=error msg="Exemplar query failed" query=4.80 err="[]v1.ExemplarQueryResult: decode slice: expect [ or n, but found \u0000, error found in #0 byte of ...||..., bigger context ...||..."

I found out, Exemplar is automatically enabled on some dasboards so where I could found it, I change it to false. Problem is, there is plenty more and dont know how to get rid of it without brute force - check every json file and fix it.

From the log, there is information, what is causing it. Can somebody explain what the logs try to say and how to fix it? Is it possible to turn it off as default?

Grafana version: 9.1.7.

2

So we replaced "exemplar": true with "exemplar": false in all dasboards but the problem persists.

Then, from logs I spot repeating queries (query=105 and query=4.80) every 5 seconds and found out which dasboard cause this logs. It was static numbers in dashboard - 105 and 4.80:

image
image

This numbers are also in logs but at first, I was thinking its just some ID of query.

Can somebody explain, what is this logs trying to describe and why does it occur with static numbers?