How to configure a single prometheus.scrape with multiple discovery.dns targets in Alloy?

Grafana Alloy
, ,
1

Hi everyone,

I’m trying to simplify my Grafana Alloy configuration for Prometheus scraping, but I’m running into some trouble.

Currently, I have something like this for each of my applications (around 60 services, so ~120 components in total with both scrape + discovery):

// Application Foo
discovery.dns "application-foo" {
  names = ["application-foo.internal."]
}
prometheus.scrape "application-foo" {
  targets    = discovery.dns.application-foo.targets
  forward_to = [prometheus.remote_write.default.receiver]
}

// Application Bar
discovery.dns "application-bar" {
  names = ["application-bar.internal."]
}
prometheus.scrape "application-bar" {
  targets    = discovery.dns.application-bar.targets
  forward_to = [prometheus.remote_write.default.receiver]
}

// Application Tess
discovery.dns "application-tess" {
  names = ["application-tess.internal."]
}
prometheus.scrape "application-tess" {
  targets    = discovery.dns.application-tess.targets
  forward_to = [prometheus.remote_write.default.receiver]
}

// Application Other
discovery.dns "application-other" {
  names = ["application-other.internal."]
}
prometheus.scrape "application-other" {
  targets    = discovery.dns.application-other.targets
  forward_to = [prometheus.remote_write.default.receiver]
}

This works sort of fine, but it feels very verbose, and I cannot take advantage of the clustering mechanic prometheus.scrape has.

Also due to the many scrape jobs it has, I think the memory gets quite high, not really a problem but still would like to figure this out.

What I would like to achieve instead is something like this:

// Application Foo
discovery.dns "application-foo" {
  names = ["application-foo.internal."]
}
// Application Bar
discovery.dns "application-bar" {
  names = ["application-bar.internal."]
}
// Application Tess
discovery.dns "application-tess" {
  names = ["application-tess.internal."]
}
// Application Other
discovery.dns "application-other" {
  names = ["application-other.internal."]
}

// Global scraper
prometheus.scrape "global" {
  targets = [
    discovery.dns.application-foo.targets,
    discovery.dns.application-bar.targets,
    discovery.dns.application-tess.targets,
    discovery.dns.application-other.targets,
  ]
  forward_to = [prometheus.remote_write.default.receiver]
}

That way, I can take advantage of Alloy’s clustering mechanism on the prometheus.scrape component without having to duplicate configs.

But when I try this, I get the following error:

config.alloy:1136:3: discovery.dns.application-foo.targets target::ConvertFrom: conversion from '[]discovery.Target' is not supported

Ive also looked into a way of using the integrated function such as array.concat but don’t think that is of much help in this case.

Is there a way to merge multiple discovery.dns targets into a single prometheus.scrape? Or am I forced to keep one scrape per service?

Any advice would be greatly appreciated! :folded_hands:

2

Try this (not tested):

targets = concat(discovery.dns.application-foo.targets, discovery.dns.application-bar.targets, discovery.dns.application-tess.targets, discovery.dns.application-other.targets)