# How do we get the standard deviation over time without labels? **URL:** \ **Category:** Grafana Loki\ **Created:** [March 27, 2023, 5:04pm UTC](https://community.grafana.com/t/how-do-we-get-the-standard-deviation-over-time-without-labels/84624 "2023-03-27T17:04:17Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![jennymals](https://avatars.discourse-cdn.com/v4/letter/j/ecae2f/32.png) [@jennymals](https://community.grafana.com/u/jennymals)\ **Post date:** [March 27, 2023, 5:04pm UTC](https://community.grafana.com/t/how-do-we-get-the-standard-deviation-over-time-without-labels/84624/1 "2023-03-27T17:04:17Z")
We have logs getting through promtail to loki There are no labels. We would like to compute the standard deviation to be used for alerting. The queries queries are as follows: > avg(stddev\_over\_time({job=“job1”}[1m]) != 0) The above did not work because we had to unwrap labels and we did not have labels. We tried the following and we got a line which is zero: > stddev(rate({job=“job1”} |= `` [$\_\_interval])) Note that avg(rate({job=“job1”} |= `` [$\_\_interval])) is returning non-zero values. ---
**Author:** ![tonyswumac](https://sea2.discourse-cdn.com/grafana/user_avatar/community.grafana.com/tonyswumac/32/22401_2.png) [@tonyswumac](https://community.grafana.com/u/tonyswumac)\ **Post date:** [March 27, 2023, 5:52pm UTC](https://community.grafana.com/t/how-do-we-get-the-standard-deviation-over-time-without-labels/84624/2 "2023-03-27T17:52:29Z")
Examples of log would be helpful. ---
**Author:** ![jennymals](https://avatars.discourse-cdn.com/v4/letter/j/ecae2f/32.png) [@jennymals](https://community.grafana.com/u/jennymals)\ **Post date:** [March 27, 2023, 6:29pm UTC](https://community.grafana.com/t/how-do-we-get-the-standard-deviation-over-time-without-labels/84624/3 "2023-03-27T18:29:50Z")
[TRACE] 2023-03-26 20:21:01.512 (DefaultId.java:84) -sftp- {“location”:”remote”,”which”:”response”,”id”:”6791723644”3412,“HTTP/1.1”,“status”:200,“headers”:{“Deny”:[“bytes”],“Cache”} [DEBUG] 2023-03-26 20:21:01.512 (Server.java:1131) -sftp- - Success 200 [INFO] 2023-03-26 20:21:01.511 (Middleware.java:454) 17981273 -sftp- Request Time: 22 ms [ERROR] 2023-03-26 20:21:01.511 (function java:56) 798164312 -sftp- Internal error occurred [DEBUG] 2023-03-26 20:21:01.511 (Modification.java:149) 109283743 -sftp- Response Time: 50 ms ---
**Author:** ![tonyswumac](https://sea2.discourse-cdn.com/grafana/user_avatar/community.grafana.com/tonyswumac/32/22401_2.png) [@tonyswumac](https://community.grafana.com/u/tonyswumac)\ **Post date:** [March 27, 2023, 6:43pm UTC](https://community.grafana.com/t/how-do-we-get-the-standard-deviation-over-time-without-labels/84624/4 "2023-03-27T18:43:51Z")
What are you trying to calculate stddev on? Request time? Response time? ---
**Author:** ![jennymals](https://avatars.discourse-cdn.com/v4/letter/j/ecae2f/32.png) [@jennymals](https://community.grafana.com/u/jennymals)\ **Post date:** [March 27, 2023, 6:49pm UTC](https://community.grafana.com/t/how-do-we-get-the-standard-deviation-over-time-without-labels/84624/5 "2023-03-27T18:49:00Z")
We are getting all the lines with errors. Then we are trying to get all the time where the count is greater than 2 standard deviation and send an alert. We will also try z-score as threshold for alert but haven’t reached that yet. ---
**Author:** ![tonyswumac](https://sea2.discourse-cdn.com/grafana/user_avatar/community.grafana.com/tonyswumac/32/22401_2.png) [@tonyswumac](https://community.grafana.com/u/tonyswumac)\ **Post date:** [March 27, 2023, 10:28pm UTC](https://community.grafana.com/t/how-do-we-get-the-standard-deviation-over-time-without-labels/84624/6 "2023-03-27T22:28:51Z")
I am not sure if it’s possible to do this with stddev. Perhaps try the `rate` function? ---
**Author:** ![jennymals](https://avatars.discourse-cdn.com/v4/letter/j/ecae2f/32.png) [@jennymals](https://community.grafana.com/u/jennymals)\ **Post date:** [March 28, 2023, 2:28am UTC](https://community.grafana.com/t/how-do-we-get-the-standard-deviation-over-time-without-labels/84624/7 "2023-03-28T02:28:05Z")
why does stddev return zero? ---
**Author:** ![jennymals](https://avatars.discourse-cdn.com/v4/letter/j/ecae2f/32.png) [@jennymals](https://community.grafana.com/u/jennymals)\ **Post date:** [March 28, 2023, 2:30am UTC](https://community.grafana.com/t/how-do-we-get-the-standard-deviation-over-time-without-labels/84624/8 "2023-03-28T02:30:42Z")
This one also is returning zero with stddev: [https://play.grafana.org/d/T512JVH7z/loki-nginx-service-mesh-json-version?orgId=1&editPanel=9](https://play.grafana.org/d/T512JVH7z/loki-nginx-service-mesh-json-version?orgId=1&editPanel=9) Query: stddev(rate({$label\_name=~“$label\_value”} |= `` [$\_\_interval])) ---
**Author:** ![tonyswumac](https://sea2.discourse-cdn.com/grafana/user_avatar/community.grafana.com/tonyswumac/32/22401_2.png) [@tonyswumac](https://community.grafana.com/u/tonyswumac)\ **Post date:** [March 28, 2023, 3:14pm UTC](https://community.grafana.com/t/how-do-we-get-the-standard-deviation-over-time-without-labels/84624/9 "2023-03-28T15:14:37Z")
The link you provided is probably not the same query you were using anymore. For your second query, you are searching for empty log line (`|= `` `) Are you sure it actually returns results? ---
**Author:** ![jennymals](https://avatars.discourse-cdn.com/v4/letter/j/ecae2f/32.png) [@jennymals](https://community.grafana.com/u/jennymals)\ **Post date:** [March 28, 2023, 3:58pm UTC](https://community.grafana.com/t/how-do-we-get-the-standard-deviation-over-time-without-labels/84624/10 "2023-03-28T15:58:07Z")
Yes, avg returns a non-zero graph. stddev returns a flat line at zero. Could you show me stddev without labels in the demo link below where stddev is not zero: [https://play.grafana.org/d/T512JVH7z/loki-nginx-service-mesh-json-version?orgId=1&editPanel=9](https://play.grafana.org/d/T512JVH7z/loki-nginx-service-mesh-json-version?orgId=1&editPanel=9) ---
**Author:** ![tonyswumac](https://sea2.discourse-cdn.com/grafana/user_avatar/community.grafana.com/tonyswumac/32/22401_2.png) [@tonyswumac](https://community.grafana.com/u/tonyswumac)\ **Post date:** [March 28, 2023, 4:18pm UTC](https://community.grafana.com/t/how-do-we-get-the-standard-deviation-over-time-without-labels/84624/11 "2023-03-28T16:18:08Z")
I think you may be misreading on what the function `stddev` and `stddev_over_time` do, and how they are different. - `stddev_over_time(unwrapped-range)`: the population standard deviation of the values in the specified interval. - `stddev`: Calculate the population standard deviation over labels So when you do something like `stddev(rate({$label_name=~“$label_value”} |= `` [$__interval]))`, you are calculating the stddev over the labels produced by `{$label_name=~“$label_value”} |= `` `, and if there is only one set of label it would of course be 0. Judging from your original post, you are looking for a sudden spike of error logs (please correct me if I am wrong). Naturally you’d probably want to do `stddev_over_time( count_over_time ( {Select for error logs} [range]) [range])`, but I don’t think you can chain two unwrapped-range functions together in LogQL, hence I recommended trying out rate so you can calculate rate of change for the number of error logs, and see if that would be sufficient. ---
**Author:** ![jennymals](https://avatars.discourse-cdn.com/v4/letter/j/ecae2f/32.png) [@jennymals](https://community.grafana.com/u/jennymals)\ **Post date:** [March 28, 2023, 4:35pm UTC](https://community.grafana.com/t/how-do-we-get-the-standard-deviation-over-time-without-labels/84624/12 "2023-03-28T16:35:31Z")
Ok, thanks. The issue we are having with rate only we are getting too many false positives. Rhe errors are frequent so we need a way to filter those with low errors and those with large errors out. We don’t know the threshold exactly. So, we thought about z-score and standard deviation to filter out those errors outside those “bands”. Do you have any ideas how to resolve the above? ---
**Author:** ![tonyswumac](https://sea2.discourse-cdn.com/grafana/user_avatar/community.grafana.com/tonyswumac/32/22401_2.png) [@tonyswumac](https://community.grafana.com/u/tonyswumac)\ **Post date:** [March 28, 2023, 7:09pm UTC](https://community.grafana.com/t/how-do-we-get-the-standard-deviation-over-time-without-labels/84624/13 "2023-03-28T19:09:16Z")
I don’t know if that’s possible without doing something more elaborate. Maybe someone with more experiences can comment. ---
**Author:** ![system](https://us1.discourse-cdn.com/grafana/original/2X/e/ef5308cc58c9e5a13187b5012eb99f406928462f.png) [@system](https://community.grafana.com/u/system)\ **Post date:** [March 27, 2024, 7:09pm UTC](https://community.grafana.com/t/how-do-we-get-the-standard-deviation-over-time-without-labels/84624/14 "2024-03-27T19:09:29Z")
This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.