Possible Bug? Multi-value variable query appears to not be working

In screenshot 1, you can see that my variable has 3 possible values. Cool.
46%20AM

In screenshot 2, I have 3 different visible queries, each using one of those 3 values. These work. Cool.
28%20AM

In screenshot 3, I have 1 visible query that I expect to show me all 3 possible values for that variable. instead, it only shows me the first one. You may also note that the query inspector shows only 1 of those 3 values are being sent in the query.
16%20AM

What can I do to see all 3 values with that query?

This is Grafana 5.2.2.

It works as expected at https://play.grafana.org/d/000000063/prometheus-templating?orgId=1 (v5.2.1).

Could it be related to the fact you’re using a “mixed datasource”?

I your screenshot only the foundation variable is using All , all others use single value. And your queries are not using any variables at all

Thanks for the responses.

I your screenshot only the foundation variable is using All , all others use single value.

I don’t understand what significance that has on my issue. Is there something wrong with that? That variable selection is intentional and says “show me the metrics on all apps with a particular name in all registered cloud foundry foundations”. I don’t think the problem relates to the variable selections.

And your queries are not using any variables at all

Not quite true. If you look at screenshot 1’s query, you’ll see that that uses each of the variables in the dashboard dropdowns.

The full query in screenshot 1 is:

label_values(cf_application_info{environment=~"$foundation",organization_name=~"$cf_organization_name", space_name=~"$cf_space_name",application_name=~"$cf_application_name"}, application_id)

This effectively says “show me all app-ids of apps with a particular env/org/space/name”. I don’t think there is an issue with this query either.

Could it be related to the fact you’re using a “mixed datasource”?

I tried it with a single datasource and had the same problem.

I’m wondering if the problem is that the query in screenshot 1 and the query screenshot 3 both use =~ to expand the variables? Does Grafana perhaps not support doing this effectively twice in a query?

=~ should be there. Are you using repeatable rows/panels?

I don’t believe so. Is this all you’d need to see to verify that?

06%20AM

But seems like you’ve only selected one applicaton - if you select more application wouldn’t that result in multiple series in graph

So the dashboard is intended to monitor cloud foundry apps. Cloud Foundry assigns a unique id to every build/deployment of an app. Therefore, any particular Cloud Foundry appname may have multiple ids associated with it (one for each build).

I’m using an exporter on Cloud Foundry to obtain metrics on the apps into Prometheus. Most of these metrics identify apps only by those unqiue app ids.

When I’m looking at metrics on a cf app in my dashboard, I’d like to be able to just select an appname and see metrics on all builds/deployments of it.

Thus, if you look back at screenshot 1, the point of that variable is to give me all the app ids for a particular appname, and then in screenshot 3, I try to query metrics with each of those app ids.

I don’t think there’s an issue with the nature of my queries. I believe there’s either a setting somewhere I’m missing, or (as I’m increasingly suspecting) there’s a bug with how Grafana is handling these queries.

I think that the problem is that your variable gets the “All” variable value, but unsure. Please remove the hide variable so that you can verify that there are multiple application ids selected in drop down and also after you’ve filtered the other variable drop downs. If you instead of your cf_application variable apply those filters directly in query I think it would work better.

I unhid the variables and everything works fine. I’m not sure why hiding them breaks the query. In any case, this works well enough for me. Thanks for the help!