Help with Complex Fill below Scenario

  • What Grafana version and what operating system are you using?
    9.2.4

  • What are you trying to achieve?
    Dashboard showing min/max bands around cpu usage line.
    Variables let me select the cluster and the systems within that cluster the data is shown for.

  • How are you trying to achieve it?

Data is from an SQL database.
Each row has: timestamp, cluster, system, min, avg, max
Variables are set up to select cluster and one or more systems within the selected cluster.
So - Cluster: isengard, Systems: gandalf, sauman (only gandalf selected)

Matching data:

monday,isengard,gandalf,5,10,15
monday,isengard,saruman,10,15,20
monday,shire,bilbo,3,4,5
monday,shire,frodo,10,15,20
tuesday,isengard,gandalf,5,10,15
tuesday,isengard,saruman,10,15,20
tuesday,shire,bilbo,3,4,5
tuesday,shire,frodo,10,15,20

The variables feed into the SQL query so it’ll only select the desired data rows.

SELECT * WHERE cluster = '$cluster' AND systems in {$systems}

Overrides set up for Min, Avg and Max.
Max fill below to Avg
Avg fill below to Min
I’ve removed the Display name overrides (as they currently break Fill To).

  • What happened?
    This works fine with gandalf selected.
    If I change the system from gandalf to saruman or I switch to a different cluster, I lose the fill to below (it shows the correct max, min and avg lines but with no fill).
    I think this is happening because the fill to below has the original system name (gandalf) coded in it (and the GUI won’t let me edit the name to substitute it with a variable).

  • What did you expect to happen?
    I expect the fill below to work regardless of which system in which cluster I select.
    With saruman selected, it should fill from max(saruman) to avg(saruman) to min(saruman). What it seems to be trying to do it to fill from max(saruman) to avg(gandalf) and from avg(saruman) to min(gandalf). The gandalf series aren’t even being pulled out of the SQL database because the dashboard variables change the SQL Query, so the fill is failing as it can’t find its target data series.

  • Can you copy/paste the configuration(s) that you are having problems with?
    No, not easily.

  • Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.
    Nope. Code ran fine, it just tried to do the wrong thing.

  • Did you follow any online instructions? If so, what is the URL?
    Patching together from here and there. Documentation is a bit ropey, especially when searching for confidence bands.
    Did find issue 43577, which prompted me to remove the display names.