Multiple Alloy configs via import file and html challenges

Grafana Alloy
, ,
1

Hi All,

I have a couple of challenges (a a newbie), though I got a lot of great thing to work in a good way:

  1. I work against three different environments, each with individual authentication methods, cert and basic.
  2. I have individual Windows and Linux configs.
  3. I have individual app/system/infrastructure configs in a directory structure
  4. While testing configs we use import file, while prod uses import html
  5. Our configs will be contributed to 500+ customers, each with individual OS, systems, infrastructure and apps to monitor.
  6. We use environment variables to set the customer, current environment to which we send the logs, path to configs, hostname etc.

I need to solve at least 2 things:

  1. By use of an env. var, say: ENDPOINT_ENV=Test, I need Alloy to import a config (file) called Test.alloy (the easy part). The challenge is what to put in Test.alloy. I’m aiming at adding all ENDPOINT_ENV relevant import (file|html). Currently we have multiple "declare “fetch” blocks in separate files - one per system/aps/infrastructure to collect information from, and this all works great - except for this:

loki.process “default” {
forward_to = argument.forward_to.value
stage.tenant {
value = string.format(“%s_sys”, coalesce(sys.env(“TENANT_PREFIX”), “cust”))
}
}

Where I have to comment the “stage.tenant” for it to work, as there is no tenant defined in Test, as one of the challenges.
We have one common config file with declare “send”", which among other controls: loki.write “default”. This also works fine except I need two different endpoints - one for certificates and one for basic_auth. Are both configured with different names, one still fails if certs can’t be found.
So what was hoping to do here is using the ENDPOINT_ENV i.e. Test.alloy to help this out, so every core and depending specific detail need for a specific env will be defined in i.e. Test.alloy, and all other imports will use the settings from this config.
So one of the questions are, can we use nested imports (I think the answer is yes)? Doing so (nested), can move to conflicting part from the current declare “send”" config file, so if define ENDPOINT_ENV=Test I can have a declare “send”" part in the Test.alloy for basic auth, vise versa a tls part in the Prod.alloy ?
Or is there a smarter way to control these quite different and dependent environment settings in a way that won’t raise conflicts in the config files, AND still be flexible?

  1. We need to find a way, where Alloy for any given customer ONLY reads(imports) configs that are relevant for the given customer. Here again I thought of using an (maybe additional) environment variable(s), which would hold a list of “things” (configs) to import. How could this be achieved in a good way? PS Octopus Deploy will be used to deploy Alloy to the customers.

Many thanks in advance.

/Bjarne