Mapping outlier values, recommended strategy/transformation

Hi I am using Grafana Cloud and InfluxDB cloud. I have some sensor data, an upon errors these sensors send a value of 99.99. I managed to map these values to show error using value mappings, as in this image:

Now this works okay, but the usual values in my panel are between 0 and 20, so having these outliers at 99.99 makes the non-error data difficult to view. One idea I got is to replace the values = 99.99 with a negative number like -0.1, so the graph looks much better.

However the only way in which I managed to achieve this is through the query itself. Using flux (the influxDB query language) I used the map function:

  |> map(fn: (r) => ({ r with _value: if r._value > 99 then float(v: "-0.1") else r._value }))

This works fine for small datasets, but if I try to view the last 30 days performance is not great, loading a visualization takes a lot of time.

I am wondering if there’s a grafana transformation I could use to achieve the same thing? I tried testing transformations and googled but was unsuccessful. Also perhaps someone has better advice as to how to show errors without distorting the whole graph.

Thanks,
Pablo

You may try:

  • logarithmic scale
  • exclude errors from “values” timeseries and select only errors (>99.9 in your case) as another timeseries + use overrides to customize it for your needs (e. g. right axis, bars, red colors with 50% trasnparency,…)
1 Like

in influxdb tags are indexed fields are not. if not designed so, things will come to a crawl. how have you designed your bucket?

I just use the influxdb library for Arduino with an ESP32. So there’s an instance of an InfluxDB point which I call sensor, and then every 5s or so the following code is called:

sensor.clearFields();
sensor.addField("Pressure NEG", DPT_E.NG);
// other fields such as pressure in other locations
published = (client.writePoint(sensor));

Not much design is into the bucket, I literally just adapted the Arduino client library example to my needs.

1 Like

As far as performance issue it might be a schema design issue (not a grafana issue)

If you can’t add a tag I would do a mass update to make those values to -0.1 and change your insert code to that value

So if you fix that it should speed things up