How hard is it to count string occurances?

Grafana Cloud
1

What I’m trying to do: count a string (‘error’) from logs over a time period and then put that number on a graph. Should be easy, right?

How I’ve failed:

  1. “input data must be a wide series but got type long”
  • happens when I try to do really any ‘expression’ things
  1. “invalid aggregation sum_over_time without unwrap”
  • happens any time I use the “_over_time” functions.
  1. Add any ‘unwrap’ to fix the above.
  • From cloudwatch, I do have one label - something like ‘_cloudwatch_owner’ but that gives me truly unbelievable numbers so there’s no way this works and I wouldn’t understand how if it did.
  • there are no other labels available on this drop down. I do have labels in the raw json view of the logs, but I can’t choose them here.

So…what am I doing wrong here?

2

I would say that Grot will be your helpfull teacher for this particular use case:

image
image1718×1172 301 KB

A bit of tunning:

sum(count_over_time({job=~".+"} |~ "error" [$__auto]))

Explore has Explain query feature, which may also help to understand final query:

image
image2062×420 43.5 KB

3

Wow. That worked! I was definitely missing the first sum and tried to go right to sum_over_time and just got kicked a bunch by grafana. Thanks!

For reference, this is the sanitized query I needed to get my data:

sum(count_over_time({instance="i-00000000000", filename="/path/to/logs/server-error.log"} |~ `(?i)error` != `[bogus/WARNING] Buggy Deprecation Notice` [$__auto]))
1 Like