Hello Everyone,
I have to configure around 30000 alert rules for OPC tags in Grafana, what would be the best way to do it? I am currently using InfluxDB as the data source.
I tried to create around 3000 alert rules using Grafana-managed alerts, but it has lots of performance issues, Grafana started to freeze and became very unresponsive.
Kindly suggest how to overcome this problem.
Thank you.
@aceankit99 have you had a chance to review the suggestions for alerting here:
Performance is a complex puzzle. Do you have any other context to share with the community about what you’ve observed with resource utilization?
- What does CPU, RAM, & disk usage look like when your instance freezes up?
- Are there any errors in your Grafana or InfluxDB logs when it freezes?
- Which database are you using? Is it local to your Grafana instance? Do you see any indication that the database or its’ resources could be a bottleneck?
- What is the frequency that your alerts are evaluating (5m, 10m, etc)? Are they all evaluating at the same time or is it staggered?
- Are all of the alerts querying the same Influx datasource? Does the datasource itself show any performance concerns?
