Best practice for using Prometheus with Alloy

Grafana Alloy
1

I’ve setup alloy to push metrics to prometheus and works ok. The issue I have is if a server dies I’m not aware. Looked at setting an alert for this but as the metrics are being pushed it does not seem to work as prometheus seems to expect it to scrape and if it can not it knows. How can I get around this so prometheus knows when my server is down? I’m using the Node Exporter Full | Grafana Labs dashboard.

I tried setting up alloy so it would collect metrics and prometheus would scrap but I can not get that working for some reason. Alloy will not create a listening port.

logging {
  level = "warn"
}

// Exporter to collect system metrics
prometheus.exporter.unix "default" {
  include_exporter_metrics = true
  disable_collectors       = ["mdadm"]
}

// Default scrape configuration to expose metrics
prometheus.scrape "default" {
  targets = [
    {
      __address__ = "0.0.0.0:9100", 
      job         = "servers",  
      platform    = "prod" ,
    }
  ]

  forward_to = [] 
}

// Local system metrics exporter
prometheus.exporter.unix "local_system" { }

// Scrape local system metrics
prometheus.scrape "scrape_metrics" {
  targets = prometheus.exporter.unix.local_system.targets
  forward_to = [] 
  scrape_interval = "10s"
  job_name = "servers"
}
1 Like