Grouping and Comparing file based events before sending alert

Grafana Alerting
1

Hello folks,

Was hoping if anyone of you have already overcome this challenge before?

I am using Google Cloud Monitoring Data Source and due to some bug in the application it is generating twice of same event

Use case is:
a file get’s created and deleted within 2 minutes of Duration is OK scenario.
However, if it takes longer and there is no delete event within 2 minutes that’s a n alerting state.

In Logs we are getting
FileName1 Create 10-02-2024 00:00:00.001
FileName1 Create 10-02-2024 00:00:00.003
FileName1 Create 10-02-2024 00:00:00.004

FileName1 Delete 10-02-2024 00:00:06

Approach we’ve used is number of events detected within 2 minutes if are less than 2 then alert should be triggered.
In this scenario, due to multiple create events we are unable to control the state

Moreover, there is no defined pattern if create will be twice or thrice.

Has anyone come across situation like this?

We don’t expect application deployment anytime soon.

Let me know if there are any questions.

2

Hi @Rohan, I am not sure if it is related to Grafana Alerting. If it is, can you please provide more information about your rule? Another possible reason is that you run Grafana in HA and alerting is not configured correctly.

3

Thanks @yuriy.tseretyan ,

My Analysis about this situation is - Since Grafana is pulling information from Google Cloud Logging - it’s making an API Call - and every time a new entry is detected, it basically treats as a new entry - However our use case requires us to group aka Transform data before an alerting state is identified, i believe Transformation does not work on alerting panels, is that corrent?