Variable with special characters (bad_data: invalid parameter "query": parse error)

Grafana Prometheus
1

Hi!

I have configured a variable for my Grafana dashboard. When the content of the variable is a simple string (e.g. “httpbin”), I can use it without any problems in the chart.

But as soon as the value contain any special characters like a space (e.g. “httpbin rate limiting”) or a minus (e.g. “httpbin-rate-limiting”), I get a parsing error in all the charts using the variable.

Error:
bad_data: invalid parameter "query": 1:16: parse error: unexpected <op:-> in grouping opts, expected "," or ")"

Here is the example for a query using the variable:

sum by ($tykapiname) (
  rate(calls{span_kind="SPAN_KIND_SERVER"}[1m])
)

How can I work with variable containing special characters?

thanks in advance,
Sonja

2

You are trying sum by label from variable. Prometheues standard has convetion for labels names:

Label names may contain ASCII letters, numbers, as well as underscores. They must match the regex [a-zA-Z_][a-zA-Z0-9_]*

As you see space, minus is not valid character for label name, so it is expexted that it will fail when you are trying to use them.

You shoulf fix your metric generation process first.

1 Like
3

thanks for the help, makes sense!

Any quick tips for me on how to process to fix the metric generation process? is using a recording rules in Prometheus the right place to do that?

4

:person_shrugging: I don’t know. You are asking out of blue and you don’t show what you are doing.

IMHO it looks like your metric generation is based on traces. So why you don’t utilise metric-generator, maybe metricsgenerationprocessor. Some trace storages have query languages (e.g. Lightstep UQL), which can create metric from the traces on the fly, so you need generate metric. Why you are using custom recording rule?

5

yes, you are right, that wasn’t very clear :sweat_smile: Sorry for that!

I’m using OpenTelemetry collector with the span metrics connector to generate the data from the OTel traces. Looks like metricsgenerationprocessor might work. I will give it a try.

Thanks for the quick help, really appreciated :hibiscus: