Hello all!
I am using Grafana v9.4.3.
My alerts are unified.
Right now I’m facing with the problem in alerts like ‘DatasourceNoData’. I keep getting these notifications everyday for 5-6 alerts in my list and after I check for them and there is data after running the query. My alerts need to send notification when there is no data at yesterday’s date.
Query, conditions are working fine, but I dont know what to do. I cant set
Alert state if no data or all values are null to OK, cause the main goal of my alerts is sending alert when there is no data. But the alerts is coming everyday and they are all wrong for some reason.
Thank you.
Also If I delete the alerts, they are still located in Groups like “Active for 5 days”.
Hi! 
If Grafana is sending NoData alerts, but re-running the query at a later date returns data, then probably the query needs to be adjusted. Are you using a Prometheus datasource?
I’m using MS SQL datasource.
Yes, of course.
select
cast(date_and_time as date),
count(id) as cnt
from
table
where
cast(date_and_time as date) = dateadd(day, -1, cast(getdate() as date))
group by
cast(date_and_time as date)
order by
cast(date_and_time as date)
Thanks! So it looks like this queries the number of rows from the previous day, is that correct? Can you explain the GROUP BY and ORDER BY, as I would have through this query only ever returns one row?
Yes, you are right.
Group by for count agg function by date_ant_time column to be able to look at date and order by was before this code was showing the previous day, there was a condition for showing last 7 days, that’s why I used order by, but now it’s just in the code. Tbh I didn’t recognise that I don’t need order by right now untill you asked me this question.
When do the DatasourceNoData errors happen? Is the data guranateed to be in this table at 00:00 when Grafana switches from querying one day to another day? Is data ever deleted from this table?
Data being inserted every day by previous day data around at 7-8 am,
DatasourceNoData error alerts are coming for the last one week ig and at the same exact time at 6am.
About deleting can’t tell surely, but I think no.
That makes sense then right? If the alert is evaluated at 6am, but the previous days data is not inserted into the database until 7-8am, then the query will return no data for ~ 2 hours until the data is inserted? Or did I mis-understand?
You are right.
Is there any chance to make alerts to get triggered a bit more later or something like that?
I’m afraid you can’t tell Grafana to evaluate the alert at set hours of the day or days of the week, it’s intended evaluate continuously and for the data you’re querying to already be present in the datasource.
What you can do is silence these alerts every day between 00:00 and 09:00 using a mute timing. Then, if you still receive a DatasourceNoData alert after 09:00 you know the data is still missing.
Sorry for the late delay and thank you for your answer!
I guess it will be helpful in my case, so I’ll try it and let you know about this case.
Was trying to create mute timing but there is a problem ‘bad request data’. I’m showing time in UTC time, I live in Kazakhstan, so I am using UTC converter to get right time, but there is a still ‘bad request data’.
any suggestions please?
Hi!
You need to add two time ranges if you want to mute across dates in UTC. For example:
There is a new user interface in Grafana 10.2 for mute timings that should make this easier to use.
1 Like
Hi again, sorry, but I have one more question about mute timing.
I was able to create mute timing without time ranges only specify days, month and year.
When I am trying to specify time range, it gives me error ‘bad request data’
You don’t need to set Days of the week, Days of the month, Months or Years.
Under Time Range, you must set a time range between 00:00 and 23:59. You should have:
18:00 -> 23:59
00:00 -> 02:00
02:05 -> 04:00
ohhhh, got it, thank you!
