How to add items into log payload in Alloy?

Grafana Alloy
,
1

TLDR

  1. I want to extract a field from the JSON log
  2. then extract value from this field via regex
  3. finally, add extracted value with a different key back to JSON log

Problem
I am receiving events from Grafana Faro SDK into Alloy and forwarding them to Loki in JSON format. When the event goes into loki.process it looks like this:

{
    "app_name": "main-app",
    "event_name": "faro.performance.resource",
    "kind": "event",
    "event_data_name": "https://cdn.company.com/specific-service-name/app_v2.23131284a8251.js",
    ... // 20+ other fields (can be dynamic)
}

From event_data_name I would like to extract the first path segment - specific-service-name and append it back to JSON with the name url_group. This must happen without modifying any other fields.

{
    "app_name": "main-app",
    "event_name": "faro.performance.resource",
    "kind": "event",
    "event_data_name": "https://cdn.company.com/specific-service-name/app_v2.23131284a8251.js",
    "url_group": "specific-service-name",
    ... // 20+ other fields (can be dynamic)
}

For some reason, in every described solution, the extracted field is added to labels, but I want to have it message payload. Is there any way to accomplish this?

2

Normally I personally prefer to not alert the source logs at all if I can help it. Is there a reason you can’t parse for that with LogQL query instead?

3

Thanks for responding!

Yes, there are two major reasons why I need to alter source data:

  1. Data is huge and varies a lot. Events from different sources will have different regex patterns and LogQL extraction will not be feasible. Additionally, queries will already take a lot of processing power so regex will further slow them down.
  2. Not all data consumers are proficient with LogQL. This data will be available to all developers and engineers, not all of them have good knowledge of LogQL or regex. Simplifying things from our side will make it easier to do visualizations.

Here is what I came up with so far:

stage.json {
  expressions = { url = "event_data_name" }
}

stage.regex {
  expression = "assets.(net|com)(?P<url_group>/.*?/)"
  source     = "url"
}

stage.replace {
  expression = "(\"event_data_name\")"
  replace    = "\"url_group\":\"{{ .url_group }}\",\"event_data_name\""
}

Definitely not the best way to edit JSON, but it works.