Both Active (yellow, the current value) and Pool size (green, the max possible value) are taken from the underlying Prometheus data, because I’m repeating this chart for multiple services and each may have a different max value.
I would like to highlight the times when a given condition occurs, specified as a formula. For example, Active > 80% of Pool size (by percent), or maybe Active > Pool size - 5 (by absolute margin).
This kind of style has been used for a long time in network monitoring software, see for example some of the GIFs produced by RRDTool and related libraries.
PS. I’m not particular about lines, if highlighting critical values is easier to do with bars, for instance by stacking or replacing the whole bar, it would be just as good.
do something like this in the override section
graph style → gradient mode = scheme
standard options → color scheme = from treshold → define your value
graph style → fill opacity = 0
indicates to me that your max value, i.e. the threshold by which you wish to color the lines, will change. For that reason, I do not think it’s possible unless you go in and define the threshold for each dataset (which maybe is not a big deal for you). The subject of dynamic thresholds has come up for a long time on these forums and it’s still not possible in Grafana.
I simply decided what my dynamic threshold should be (in my case, 80% of Pool size) and I wrote a PromQL expression that returns the Active value only when it’s above the threshold; null (not zero) otherwise.
My series now are:
dbpool_size as Pool size
dbpool_active as Active
dbpool_active and dbpool_active > 0.8 * dbpool_size as Critical
(All vector usages above have {instance=~"$instance"} after them, omitted for brevity.)
I tried various styles, including a hidden series with just the threshold (the part after and) to be used in Critical with the Fill Below option. But I decided that a big red bar is more visible.
Just to be clear, how are you defining the 20 in you graph? Like my screenshot below? If yes, then you’ll have to change it, right? That’s what I meant by dynamic not being available.
No, they are not static. Both the maximum value (20 above) and the threshold which is used to compute the red critical bars (in my case: 80% of 20 = 16) are taken straight from the data source and can be change instance by instance and from time to time.
To reiterate, my data source has 2 source vectors, one gives me the current value of the resource (dbpool_active) the other gives me the maximum configured value for that instance (dbpool_size).
In my chart I have set the Y axis to some large fixed value N, one that I know covers all possible cases, in my case 100. This allows me to visually compare all the repetitions of the chart, because they have the same scale.
As I said, the real maximum for each instance is taken from its own data vector because it’s a dynamic value: it can change if I change the docker instance configuration. For example, I recently changed one instance from 10 to 20 (the active value remained around zero because that instance was unused at the time):
The threshold that produces red bars is computed from this max value, using the formula I wrote in PromQL (in my case, 80% of max) so it’s completely dynamic: it changes from instance to instance and from time to time.
Here I’m plotting the threshold used to compute the critical bars, as you can see it follows 80% of the max value:
Got it. So just for those following along, the threshold you have created & displayed comes from your own query, which you called “Threshold”. The Threshold that Grafana offers (and which I showed in my screenshot) is static and cannot be tied to a query.
Hi, I’m struggling to get this type of threshold display as the threshold cannot be dynamic. I have a metric from which I’m reading the count of no of ports for a particular system. I want to put a threshold value which I can get from another query. How can I do that? Could anyone help me with the queries and config to achieve this? Attaching a screenshot of the bar graph on which I need the dynamic threshold.
