Structured metadata missing in Grafana

[jelmerderonde1]

Hi, I’ve just started using Alloy in combination with Loki, but I have trouble getting structured metadata to work. My setup:

• Alloy config is as in the image.
  

  

• I use discovery.relabel to relabel some low-cardinality labels, like namespace and env, as I understand it, everything I don’t relabel will not be written to loki because the label name will start with __.

• In loki.process I have a stage.structured_metadata that takes some (higher-cardinality) labels and adds them to structured metadata, stuff like pod_name.

• I get no errors and the labels are showing up in Grafana as expected. However, the structured metadata is missing.
  Some snippets from my Loki config:

  schemaConfig:
    configs:
    - from: "2024-01-01"
      store: tsdb
      index:
        prefix: index_
        period: 24h
      object_store: 'swift'
      schema: v13

  limits_config:
    allow_structured_metadata: true
    volume_enabled: true
    retention_period: 740h

• I’m running Alloy v1.7.1, Loki 3.4.2 and Grafana 11.5.2.
• I’ve looked at the data that Alloy sends to Loki (using tcpdump and wireshark), and as far as I can tell the structured metadata is not sent. I couldn’t get the protobuf messages to decode, but just looking at the ASCII data the structured metadata is not there.

What am I missing? Any help is appreciated!