Add additional index labels in Loki 3.0 via OTLP

Grafana Loki
,
1

Hi everyone,

I use Loki 3.0 and ingest logs in OTLP format with the OpenTelemetry Collector and the otlphttp exporter.
Everything works as expected, but I cannot add the resource attribute host.name as an additional index label. The attribute is instead added as structured metadata.

I followed the instructions here: Ingesting logs to Loki using OpenTelemetry Collector | Grafana Loki documentation

Other resource attributes from the list of default labels like deployment.environment are converted to index labels as expected.

This is the section from Loki’s configuration file:

limits_config:
  otlp_config:
    resource_attributes:
      attributes_config:
        - action: index_label
          attributes:
            - host.name

In the OpenTelemetry Collector debug log, the log record that is sent to Loki looks like this:

2024-05-07T15:48:48.551+0200        info        ResourceLog #0
Resource SchemaURL: https://opentelemetry.io/schemas/1.6.1
Resource attributes:
     -> service.namespace: Str(demo-namespace)
     -> service.name: Str(demo-service)
     -> host.name: Str(example.com)
     -> deployment.environment: Str(test)
ScopeLogs #0
ScopeLogs SchemaURL:
InstrumentationScope
LogRecord #0
ObservedTimestamp: 2024-05-07 13:48:47.895773234 +0000 UTC
Timestamp: 2024-05-07 13:48:47.701 +0000 UTC
SeverityText: DEBUG
SeverityNumber: Debug(5)
Body: Str(log content)
Attributes:
     -> log.file.name: Str(log.ndjson)
     -> log.file.path: Str(/path/to/logs/log.ndjson)
Trace ID: dac9c66b486632ce141cbe9a98712c41
Span ID: 4b9cf39a00f42ce9
Flags: 1

What could I do to store host.name as an index label instead of structured metadata?

Thank you for your help!
Stephan

2

I’ve not tried using otlp collector with Loki, but according to documentation there is a list of fixed attributes that get mapped to labels. See Ingesting logs to Loki using OpenTelemetry Collector | Grafana Loki documentation

3

Thanks for you response!
I followed the documentation, there is a section about how to extend the built-in list of attributes:

But these settings do not seem to have any effect in my installation. I also tried to disable the default attributes as described here, but it also did not work:

4

I’m having the same problem. As a workaround I just changed the default list of resource attributes that gets mapped via the distributor config, but that’s not ideal. Example:

loki:
  distributor:
    otlp_config:
      default_resource_attributes_as_index_labels:
        - foo
        - bar
5

There was a bug that was fixed in PR: fix: fix parsing of default per tenant otlp config by sandeepsukhani · Pull Request #12836 · grafana/loki · GitHub. The fix has been released with 3.1.1. Please let us know if you still an issue after the upgrade.

6

Hi!
I tried adding the otlp_config inside the limits_config on loki version 3.1.1, but it is still not working.
All fields stay as resource attributes.

For the above example, should this work?

limits_config:
  otlp_config:
    resource_attributes:
      attributes_config:
        - action: index_label
          attributes:
            - host.name