Abnormal peaks in time series - how to avoid them?

When displaying time series from influxdb i often get abnormal peaks of values exceeding the regular ones by thousand and more times. This is caused by some issues on the network or systems.
How do you usually avoid this? The image below displays an example.

Depending on what your querry is, perhaps you could include a and _value < Treshold_value.

That probably makes sense.
Below is sample with query, but i do not quite understand how to proper reference a result of the query. Putting just … AND _value < 10000000000000 … gives no data.

Not sure how to do that in influxql when using derivative functions. Tried some sub queries, but failed in my attempts. But if you have the opportunity to switch to flux, this here works:

 |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r._measurement == "wwan0" and
 r._field == "rx_bytes" and
 r.serial_number =~ /^$serial$/ 
|> aggregateWindow(every: v.windowPeriod, fn: last)
|> derivative(nonNegative: true)
|> filter(fn: (r) => r._value < 1024*1024)
|> map(fn:(r) => ({ _time:r._time,rx:r._value * 8.0}))

The key here is the |> filter(fn: (r) => r._value < 1024*1024) line. Since the data in my example is bytes I here cap at one megabyte/s.

Without cap

With cap

And as I posted the flux solution I remembered how to do it in influxql as well. Had the order wrong in my initial attempts.

SELECT max("rxbytes")  FROM 
	(SELECT non_negative_derivative(mean("rx_bytes"), 1s) *8 as rxbytes FROM "wwan0" WHERE ("serial_number" =~ /^$serial$/) AND $timeFilter GROUP BY time(10s) fill(null))
where "rxbytes" < 10000000 GROUP BY time($__interval)

The inner select is what you have already, but you perhaps need to select different time intervals in the inner GROUP BY.

1 Like

Yup, that works with influxql.
Many thanks for your help, my friend))

1 Like

Also (in case you did not know about it), Grafana introduced soft min and max to address this exact type of issue. Change axis display | Grafana Labs

1 Like

It is good to also know about this way to solve the issue. Kudos for recommending it.

That being said, personally I’m not so found of “hybrid solutions”. Meaning doing part of the limits/maths/joins in the graphing tool and parts in the query tool. The reason is that I find it hard to debug later on. Meaning when I inherit someone else’s graphs or come back to mine a year later. Remembering to not only look at the query, but also hunt around in the gui throws me off every time.

BTW, is it possible to sum two or more time series with influxql (or may be flux which i have not upgraded to yet) rather than stacking graphs or executing an expressions on them ?
For example two metrics - for uplink and downlink traffic from the examples above?

Suggest you open a new topic so its easier to search questions/answers for other forum users. :slight_smile: