Loki Variable not being loaded in Dashboard

[juanjocerero]

• What Grafana version and what operating system are you using?

Grafana v12.2.0 (92f1fba9b4) over Debian 13 Trixie. Grafana is running as a Docker image.

• What are you trying to achieve?

To modify the value of a Variable so that it correctly loads the Loki logs for the service Caddy.

• How are you trying to achieve it?

I’m editing the query value inside the Variables panel in the Dashboard. This is the Dashboard I’m currently trying to use: GitHub - Malfhas/caddy-grafana: Monitoring Caddy Server with Grafana (Prometheus + Loki) on Debian

• What happened?

The preview section showed caddy correctly. However, when I click the Run query button, nothing happens; I checked the Network tab in my browser and nothing is happening there. When going back to the Variables panel in the Dashboard settings, the Definition value is empty.

• What did you expect to happen?

I expected the logs to be shown in the proper tab in the Dashboard. I don’t know where the problem is, since using the same query in the Explore tab works fine.

• Can you copy/paste the configuration(s) that you are having problems with?

        "definition": "",
        "name": "logfile",
        "options": [],
        "query": {
          "label": "service_name",
          "refId": "LokiVariableQueryEditor-VariableQuery",
          "stream": "{service_name=\"caddy\"}",
          "type": 1
        },
        "refresh": 1,
        "regex": "",
        "type": "query"

• Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.

No errors in Grafana UI or in related logs:

1759920565787	2025-10-08T10:49:25.787Z	logger=tsdb.loki endpoint=queryData pluginId=loki dsName=loki dsUID=bf0d1pawx416ob uname=admin fromAlert=false t=2025-10-08T10:49:25.787030019Z level=info msg="Response received from loki" duration=7.698401ms stage=databaseRequest statusCode=200 contentLength= start=2025-10-08T10:44:25.652Z end=2025-10-08T10:49:25.652Z step=500ms query="{service_name=\"caddy\"}     | json  | logfmt | drop __error__, __error_details__  " queryType=range direction=backward maxLines=1000 supportingQueryType=grafana-lokiexplore-app lokiHost=loki:3100 lokiPath=/loki/api/v1/query_range status=ok

• Did you follow any online instructions? If so, what is the URL?

Didn’t follow any specific instructions, just the suggested values by Grafana. However, since it works in the Explore tab, I think it should work inside the Dashboard too, but I might be wrong.

Thanks in advance for any help!

[galenkistler]

The query is already run to show the preview values, it will be automatically run in the associated dashboard when the time range or dashboard panel queries are changed as well.

Also those queries cache, so if it already has the values available in the UI it won’t run again until you change the query in a way that could cause it to return different results.

Label values will return the values for the selected “label” filtered by the stream selector. Since you are selecting label values with a label of “service_name” and a service_name value of “caddy” in the stream selector your variable will only ever return you a single value of “caddy”.

Are you seeing any unexpected or broken behavior with the variable in the dashboard?

[juanjocerero]

Yes, it’s completely broken, the log panel is showing absolutely nothing.

Even manually running the query, with or without the transformation parameters, just seems to fail silently.

I think this is the complete log of a query run of the Dashboard:

logger=tsdb.loki endpoint=callResource pluginId=loki dsName=loki dsUID=bf0d1pawx416ob uname=admin t=2025-10-08T14:49:21.378940539Z level=info msg="Response received from loki" status=ok statusCode=200 contentLength=135 duration=4.1154ms contentEncoding= stage=databaseRequest

logger=tsdb.loki endpoint=callResource pluginId=loki dsName=loki dsUID=bf0d1pawx416ob uname=admin t=2025-10-08T14:49:21.368227106Z level=info msg="Response received from loki" status=ok statusCode=200 contentLength= duration=4.858902ms contentEncoding= stage=databaseRequest

logger=tsdb.loki endpoint=queryData pluginId=loki dsName=loki dsUID=bf0d1pawx416ob uname=admin fromAlert=false t=2025-10-08T14:49:14.226603553Z level=info msg="Response parsed from loki" duration=173.83µs framesLength=1 stage=parseResponse

logger=tsdb.loki endpoint=queryData pluginId=loki dsName=loki dsUID=bf0d1pawx416ob uname=admin fromAlert=false t=2025-10-08T14:49:14.226347977Z level=info msg="Response received from loki" duration=66.664412ms stage=databaseRequest statusCode=200 contentLength= start=2025-10-07T14:25:43.123Z end=2025-10-08T14:25:43.123Z step=1m0s query="{filename=\"caddy\"} | json | line_format \"{{.status}}\"" queryType=range direction=backward maxLines=1000 supportingQueryType=none lokiHost=loki:3100 lokiPath=/loki/api/v1/query_range status=ok

I’m basically lost at this point and have tried everything from recreating the variable to creating a new one and redirecting the panel query to use that one, and the result is always the same.

However, if I create a new panel with this query: {service_name="caddy"} | json | line_format "{{.status}}", it works correctly. I guess I can just recreate the whole thing manually, I just want to know if what’s happening is my fault or I should report this as a bug/issue on the Issue tracker on Github.

[galenkistler]

What happens if you manually create a query of {filename=”caddy”}? That’s what you’re showing in the screenshot above?

I’m a bit confused why you’re taking values from service_name and dropping them into the filename label?

[juanjocerero]

Nothing happens if I do that, I get an empty panel. The Network tab shows a query being made and a 200 response; the result contains a data.values object composed of 6 empty arrays.

That thing I posted was just me trying everything I could think of to make it work (which I somehow did); I’m not an expert at this so I’m just throwing stuff at the wall and seeing what sticks, sorry if I’m doing anything wrong.

What I was showing is different, since I’m manually enabling Docker images to be ‘scraped’ by Loki using Docker labels.