How to transform Collectd, disk_time value (from disk plugin) in something more readable

Since that number always grow and the fact that we cannot just divide it by disk_ops to get it,
what would be the easiest way to get the real disk response time in milliseconds
using Grafana functions?

I tried both options “perSecond()”, “nonNegativeDerivative” also both together but nothing to do, I always get funny numbers…

Did you figure this out?

Looking at this dashboard on Grafana.com:

https://grafana.com/dashboards/569

It is for InfluxDB rather than Graphite but the query looks like this:

SELECT non_negative_derivative(mean("value"), 1s)  / 1000 FROM "disk_io_time" 

Hard to find proper docs for this but found this:

https://github.com/signalfx/integrations/blob/master/collectd-disk/docs/derive.disk_time.read.md

So it seems the unit depends on your OS.

Hi Daniel, I still didn’t find a solution but found what cause it. When a metric is missing for whatever reason, the next one is out of space.

We are collecting data per minute so if there is no data for one minute, the next minute will pass for example from 20 to 65 million and get back to 20 after causing huge spikes in graphs…

I don’t know if it is a collectd/graphite problem but would appreciate if there would be a way to bypass it with Grafana.

aliasByNode(nonNegativeDerivative(servers.$Serveur.disk.sda.disk_time.write), 3)
Best regards,
Gervais

There are probably several ways to handle this. One way would be to use the keepLastValue function. See this StackOverflow answer for an example.

Other options:

  • movingAverage could work if you group by 3m or 5m. You can specify xFilesFactor as a parameter too.
  • if you use avgSeries then I think the xFilesFactor will apply and handle nulls as long as there are not too many nulls.
  • There are probably more options - Graphite has a lot of functions!