Trying to configure an alert such that if the query returns a value > 0, an alert should fire. My data source is of postgresql.
The evaluation interval is configured as 1d.
Pending period is None. I want the alert to be checked once a day and fire immediately if the value > 0.
The contact point is a webhook. I have a Linux pc running flask app on a port to receive the alert and do some action based on data received due to the alert fire at the end point.
Seeing that I get the webhook alert every 4 hours instead of 1 day and in the webhook response it shows the previous day’s value till evening 5:30 pm local time.
Ex- values: {A: 1, B: 1} where A is the query which returns the value. The expected value is 0 for the current day and previous day was 1.
How to get the alert to be evaluated once daily with the right value?
Hi @shruthishankaran10,
4 hours is the default value of the repeat interval option.
This causes Grafana to resend the alert notification every 4 hours if the alert is still firing (i.e., not resolved).
Docs for more context:
You could also keep the alert evaluation more frequent and use the repeat interval to control notification timing.
@pepecano thank you for the info. This would be helpful.
Was also seeing that evaluation for current day starts at 5.30 pm local time and till then it sends previous day’s value. My timezone is 5.30 hrs ahead of utc. I tried setting the default grafana timezone to both local and utc, and observed the same behaviour.
Any suggestion on how I can make it send the current day’s data?
It’s likely a query issue. PostgreSQL date functions like CURRENT_DATE may use the database server’s timezone. It’s a good practice to write alert queries using time filters in UTC, for example:
(now() AT TIME ZONE 'UTC')
It sounds like there might be a mismatch between your alert configuration and the evaluation interval. Since you want the alert to be evaluated once per day, but you’re receiving it every 4 hours instead, it’s likely due to how your alerting system is set up in relation to the PostgreSQL query and the time zone settings. First, make sure that the time zone for your data source (PostgreSQL) and the alerting system are aligned correctly, as discrepancies here could cause the alert to trigger based on UTC or a different time zone. Additionally, the “pending period” setting might need to be configured differently. Set the “pending period” to a value like “24h” to ensure that the alert is only evaluated once every 24 hours, starting from the time of the last evaluation. Also, check if the webhook contact point is set to fire only when the value changes, as you don’t want it triggering multiple times within the same day if the value hasn’t changed. This way, the alert will fire correctly based on the data returned for that day.
