We are currently facing a challenge with our setup where we fetch details from the actuator URL and post them to Grafana Cloud via Grafana Alloy. Below is the configuration we are using:
prometheus.scrape “custom_metrics” {
targets = [{“address” = “localhost:8080”}]
forward_to = [prometheus.relabel.custom_metrics_sample.receiver]
metrics_path = “/actuator/prometheus”
scrape_interval = “60s”
}
The actuator/prometheus URL exposes multiple custom metrics, such as:
status{component=“sample”,datacenter=“unknown”,environment=“abc1”,} 1.0
However, when we stop our application, this endpoint stops working, and we encounter the following error in Grafana Alloy:
health: “down”
labels:
{
instance = “localhost:8080”,
job = “prometheus.scrape.custom_metrics”,
}
last_error:
“Get "http://localhost:8080/actuator/prometheus\”: dial tcp [::1]:8080: connectex: No connection could be made because the target machine actively refused it."
Our requirement is to have Grafana Cloud insert a value of 0 into the panel if the value of the metric is missing or null. This is crucial for scenarios where a server or service goes down, and the value for that server becomes null.
In our previous dashboard queries using InfluxDB as the data source, we achieved this with the following query:
SELECT last(“status”) FROM “status_metrics” WHERE (“environment” =~ /^abc1$/) AND $timeFilter GROUP BY time($__interval), “environment” fill(0)
We need a similar query or configuration in Prometheus to achieve the same functionality. Specifically, if the metrics stop publishing to the actuator URL via Grafana Alloy, the previous metrics should be displayed as 0.
Could you please provide guidance or a solution to accomplish this in Prometheus?
We tried several solution’s but it didn’t worked for us .