Repeating row variable not working

Not sure what I’m doing wrong, but I cannot get a repeating row variable to work…

I’m using Grafana 8.4.4 via docker, and have set-up a repeating row on a variable (node).

Using that repeating row variable in the panel title (${node}) works, but using it in the query itself returns no results.

If I replace node_cpu_seconds_total{instance="${node}"} with node_cpu_seconds_total{instance="192.168.30.103:9100"}, then the query works, so I’m not sure why it isn’t picking up the desired result when using ${node}.

Maybe try this

wmi_system_threads{instance=~"$server"

Or

Can you check the following out

https://play.grafana.org/d/000000063/prometheus-templating?orgId=1&var-jobs=alertmanager&var-jobs=node&var-jobs=prometheus&var-disk_series=instance_device:node_disk_io_time_seconds:rate5m&var-query=up{env%3D"demo",instance%3D"demo.do.prometheus.io:9093",job%3D"alertmanager"}%201%201670819526000&var-instance=demo.do.prometheus.io:9100&var-instance2=demo.do.prometheus.io:9093&var-last=node

Huh, weird, node_cpu_seconds_total{instance=~"$node"} did indeed work…

Why would that work when a normal comparison operator (node_cpu_seconds_total{instance="$node"}) doesn’t?

Wrapping it in quotes for the title, there isn’t any additional spacing or anything…

Thank you :slight_smile:

1 Like

I believe dofferent data source do the variable filtering differently, which is wack

Ah I see!

Thank you again!

This thread is marked as solved, but it isn’t really. I found the explanation and the correct solution behind this problem.

Root cause: repeating variables are escaped by Grafana.

For example, if your query contains an IP address, it will be sent by Grafana as 127\.0\.1\.1 instead of 127.0.0.1. Thus, the workaround =~ works because you query is being interpreted as a regex certainly.

Yes, it’s a strange default behaviour and that’s why we are here now. Digging it in deeper, I discovered that glob formatter is being applied during variable interpolation as fallback method (Why? I don’t know). See [1] and [2] for more information.

So, how can we solve this without using regex operator =~?

The answer is to explicitly use raw variable interpolation syntax. Using as example the query in the first post, we have to rewrite the query like this:

wmi_system_threads{instance="${server:raw}")

Henceforth, my recommendation is always using explicit raw variable interpolation syntax for your repeating queries.

[1] Variable syntax | Grafana documentation
[2] https://play.grafana.org/d/cJtIfcWiz/template-variable-formatting-options?orgId=1