Please help. Simple alert use case. Can't figure it out

loki version: v2.6.1
grafana version: 9.0.7

I’m trying to implement a simple use case:
If the string ‘ERROR [’ appears in any of the logs then send an alert notification.
Don’t send notifications for this alert rule for the next 5 minutes
After 5 minutes if the string appears again then send an alert notificaions. (then don’t send for 5 minutes)

At the moment the problem I’m having is that most of the time the query of the alert rule returns NoData - so the rule always in active NoDataFromSource state.
I couldn’t find anything in the documentation about how to change the query so it doesn’t return NoData.

Please help me implement this simple use case. Thanks in advance.

Sincerely,
Michael

What is the query being used in this context ? please share more details so i can comment along with inputs .

Dear Suresh,

Thanks for getting back to me!
I’ve attached a screenshot of the alert rule query and all other properties.

Sincerely,
Michael

Hi ok so you are looking for “Error [” ,
What do you get when you try lookign for “Error”

count_over_time({host=“riot”} |=“Website” [3m]) - i use this in raw query windows and it works for me .
Also i see ’ this in range window of your screen shot , it shoudnt be there .

Dear Suresh,

Thanks for your help! I truly appreciate it.
I removed the excess back-tick and change the pattern to ‘ERROR’ however the result is still NoData as you can observe in the attached screenshot.
I think in your case there are data points for the given period. In my case there are none. I don’t want it to fire alerts when there is NoData but only when it finds ‘ERROR’ in the logs.
Feels like I’m stumbling in the dark. It seemed to me that my use case is the most common one anyone could have, am I completely in the wrong direction?

This is my use case:
if error-in-logs then fire-alert

Thanks,
Michael

ok , in that case how do you want it to be when there is no data ?
you will get such option in 2nd option(Alert evaluation behaviour) , check below screen shot.
I believe you can use any one from the drop down

Thanks Suresh!!! Your solution seems to work!

Glad it worked :slight_smile: :slight_smile: :slight_smile: :slight_smile:

@suresh300567 please tell me what I’m doing wrong. :cry:
I thought that it finally works but instead it’s giving new errors now. I feel devastated trying to setup Grafana alerts with loki for months now.


Please try to change as per your query output , from what i see its better to use Expression with classic condition.