Grafana version - v8.3.10
Operating system - Windows 10
Using Prometheus & Grafana to monitor Airflow DAGs. Want to get only the last run status of DAG, Whether like failed or successful.
Trying to achieve last run status of DAGs, by the metrics provided by Airflow and querying them in Grafana (Prometheus as Data Source)
Airflow metrics:
For Success - dagrun.duration (success)
For Failure - dagrun.duration.failed
Queries used in Grafana:
For Failure - count(airflow_dagrun_failed) by (dag_id, kubernetes_namespace)
For Success - count(airflow_dagrun_duration) by (dag_id, kubernetes_namespace)
Issue is that some of the DAGs appear in failed and success both, as they run more than once in a specific time, like hourly or daily. We are using this to monitor failed DAGs, but they appear in both success and failure so it becomes difficult to monitor.
Expecting to get only the last run status of a DAG, failed or success, using Prometheus & Grafana.