On our grafana dashboards we have queries run using template variables, such as:
program_status{$selector,region="$region",location="$location",session="$session",program_name="$programName"}
We noticed that the query is not returning any results when there are parentheses in label values. For example, when programName is FXHD(Pacific), the query does not return any results. The following is the URL used by grafana, as reported by the Query Inspector. So grafana is attempting to escape the parenthesis using %5C%5C :
“api/datasources/proxy/21/api/v1/query_range?query=program_status%7Bapp%3D%22encryptor%22%2Cregion%3D%22testRegion%22%2Clocation%3D%22testLocation%22%2Csession%3D%22testSession%22%2Cprogram_name%3D%22FXHD%5C%5C(Pacific%5C%5C)%22%7D&start=1579016640&end=1579016700&step=30”
When there are no parenthesis in the label value, query successfully returns results.
I also noticed that the same query with the same label value works fine when using the prometheus expression browser. And the expression browser is NOT using %5C%5C to escape the parenthesis.
For further testing, I also tried using curl & HTTP API of Prometheus to perform the queries directly (outside grafana) and reproduced the problem. When using %5C%5C before the parenthesis in label value, queried fail. When they are not used, queries are successful.
Could this be an issue in Grafana or a Prometheus issue? Grafana is using %5C%5C before each parentheses in label value in its queries, but Prometheus does not like them (and works fine when %5C%5C is not used before parentheses in label value).