I’ve got a dashboard that helps me monitoring the “executors” in my JVM – if you’re not familiar with them they are basically queues of work. The dashboard has two variables I often care to pivot about:
- instance (JVM)
- name of executor
Sometimes I want to key in on a specific executor (ex: “foo-processor”) and see charts that break performance out by instance, so I can use a query like this:
sum(rate(executor_seconds_sum{instance=~"$instance",name=~"$name"}[5m])) by (instance)
/
sum(rate(executor_seconds_count{instance=~"$instance",name=~"$name"}[5m])) by (instance)
With the legend set to {{instance}}.
Other times I want to narrow in on a single instance and look at all the executors it is running, so this query is what I’d use:
sum(rate(executor_seconds_sum{instance=~"$instance",name=~"$name"}[5m])) by (name)
/
sum(rate(executor_seconds_count{instance=~"$instance",name=~"$name"}[5m])) by (name)
With the legend set to {{name}}.
What I’d love to be able to do is combined this into a single panel that dynamically changed depending on whether I had selected a single instance or a single executor selected up top. I’m wondering what techniques people use for this kind of dynamic pivoting around two different dimensions.
I’ve looked at the docs and I don’t believe it is possible to adjust the “by (xxx)” part of my PromQL and the “{{xxx}” of my legend based on template variables, but perhaps I’m mistake or there is some other approach I should be considering?
Thanks!