Grafana Alerts for Custom Metrics

Hi guys! :smiling_face_with_three_hearts:

I am new to Grafana and I’m trying to get alert notifications working for some custom application metrics I’m tracking (like how long requests take and how often they error out). I can see the data just fine in Grafana, but setting up the alerts is proving tricky.

Here’s my setup: :point_right:

  • Data source: Prometheus
  • Metrics: Custom application performance metrics (latency, error rates)
  • Goal: Email alerts when thresholds are crossed (e.g., latency above 200ms)

I have been through the Grafana documentation and tried to follow the alert rule setup steps, but the alerts just aren’t firing as expected.

Hoping you all can help with a few questions:

  • Any tips for setting good thresholds for custom metrics?
  • What are common mistakes to avoid when building alert rules?
  • How can I test my alerts before relying on them in production?
  • Can I customize alert emails to include more details about the triggered metric?

I also check this document: https://grafana.com/docs/grafana/latest/alerting/set-up/provision-alerting-resoulookerrces/file-provisioning/ But I have not found any solution. Any advice, examples, or resources would be a huge help! Thanks a bunch in advance for your time!

Respected community member :smiling_face_with_three_hearts:

Can you share the screenshots of how you have configured your alert?