Traces receiver behind reverse proxy

Grafana Alloy
, ,
1

Bonjour :waving_hand:

Alloy: `v1.12.0`

Traefik: `3.6.4`

I have an Alloy receiver on VPS. And all others Alloy send to it logs, metrics, traces…

On my VPS who host Alloy receiver. I have Traefik who manage DNS with basicauth middleware to receive traces. something like: https://alloyfortraces.domain.tld

traefik.http.routers.alloytraces.rule: Host(`alloyfortraces.domain.tld`)

traefik.http.routers.alloytraces.entrypoints: https

traefik.http.routers.alloytraces.middlewares: default@file,alloyreceivertracesauth

traefik.http.routers.alloytraces.tls: true

traefik.http.routers.alloytraces.tls.certresolver: myresolver

traefik.http.routers.alloytraces.service: alloytraces

traefik.http.services.alloytraces.loadbalancer.server.port: 4317

traefik.http.middlewares.alloyreceivertracesauth.basicauth.users: traces:xxxxxxxxxx

But I have this error on all VPS with an Alloy sender instance:

{“ts”:“2025-12-16T09:57:19.036700838Z”,“level”:“info”,“msg”:“Exporting failed. Will retry the request after interval.”,“component_path”:“/”,“component_id”:“otelcol.exporter.otlp.tempo”,“error”:“rpc error: code = Unavailable desc = connection error: desc = \“transport: Error while dialing: dial tcp: address traces.domain.tld: missing port in address\””,“interval”:“15.819726413s”}

Alloy traces receiver configuration:


otelcol.receiver.otlp "default" {
  http {}
  grpc {}
  output {
    metrics = [otelcol.processor.batch.default.input]
    logs    = [otelcol.processor.batch.default.input]
    traces  = [otelcol.processor.batch.default.input]
  }
}

otelcol.processor.batch "default" {
  output {
    metrics = [otelcol.exporter.otlp.tempo.input]
    logs    = [otelcol.exporter.otlp.tempo.input]
    traces  = [otelcol.exporter.otlp.tempo.input]
  }
}

otelcol.exporter.otlp "tempo" {
  client {
    endpoint = "tempo:4317"

    tls {
      insecure = true
    }
  }
}

And Alloy traces sender configuration:


otelcol.receiver.otlp "default" {
  http {}
  grpc {}
  output {
    metrics = [otelcol.processor.batch.default.input]
    logs    = [otelcol.processor.batch.default.input]
    traces  = [otelcol.processor.batch.default.input]
  }
}

otelcol.processor.batch "default" {
  output {
    metrics = [otelcol.exporter.otlp.tempo.input]
    logs    = [otelcol.exporter.otlp.tempo.input]
    traces  = [otelcol.exporter.otlp.tempo.input]
  }
}

otelcol.exporter.otlp "tempo" {
  client {
    endpoint = "https://alloyfortraces.domain.tld"
        auth     = otelcol.auth.basic.xxx.handler
  }
}

otelcol.auth.basic "xxx" {
    username = "traces"
    password = "xxxx"
}

On localhost with Alloy sender and receiver at same time, no problem.

Same for metrics and logs, no problem behind reverse proxy.

Why needed of port for traces? I missed something in configuraion, what?

Thanks!

2

I found solution.

In traefik, I put the wrong port. The right port is 4318.

traefik.http.services.alloytraces.loadbalancer.server.port: 4318

In tempo configuration, I changed somes settings.

# Configure the server block.
server:
  # Listen for all incoming requests on port 3200.
  http_listen_port: 3200
  http_listen_address: "0.0.0.0"

# The distributor receives incoming trace span data for the system.
distributor:
  receivers: # This configuration will listen on all ports and protocols that tempo is capable of.
    otlp:
      protocols:
        http:
          endpoint: "0.0.0.0:4318"
        grpc:
          endpoint: "tempo:4317"

I changed

http_listen_address: "0.0.0.0"

and

 endpoint: "0.0.0.0:4318"

To listen on every interface, to be reached by Traefik

I don’t change Alloy receiver configuration.

In Alloy sender configuration.

otelcol.receiver.otlp "default" {
  http {}
  grpc {}
  output {
    metrics = [otelcol.processor.batch.default.input]
    logs    = [otelcol.processor.batch.default.input]
    traces  = [otelcol.processor.batch.default.input]
  }
}

otelcol.processor.batch "default" {
  output {
    metrics = [otelcol.exporter.otlphttp.tempo.input]
    logs    = [otelcol.exporter.otlphttp.tempo.input]
    traces  = [otelcol.exporter.otlphttp.tempo.input]
  }
}

otelcol.exporter.otlphttp "tempo" {
  client {
    endpoint = "https://alloyfortraces.domain.tld"
        auth     = otelcol.auth.basic.xxx.handler
  }
}

otelcol.auth.basic "xxx" {
    username = "traces"
    password = "xxxx"
}

I changed

otelcol.exporter.otlp “tempo”

to

otelcol.exporter.otlphttp "tempo

I found in documentation, otlphttp to solve the problem.

Et voilà!