Unable to see traces in Grafana via Otel collector and Tempo setup

Grafana Tempo
1

I set up an otel collector, tempo and grafana in my docker. I dont see any traces in grafana. I dont see any logs in my otel collector either so not sure if my traces are being received in otel collector. Can someone please help me with my docker compose setup if im missing something? My containers are able to startup fine.

The reason i set the port to 4319 in tempo is because i was getting an error in my tempo container saying 4317 is already in use, presumably by my otel-collector

otel-collector.yaml:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

exporters:
  otlp:
    endpoint: tempo:4319  # Tempo container name and gRPC port
    tls:
      insecure: true
  debug:
    verbosity: detailed  # Use 'normal' or 'detailed'

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [otlp, debug]

tempo.yaml:

auth_enabled: false

server:
  http_listen_port: 3200
  grpc_listen_port: 4319

distributor:
  receivers:
    otlp:
      protocols:
        grpc:
        http:

ingester:
  trace_idle_period: 10s
  max_block_duration: 5m

compactor:
  compaction:
    block_retention: 1h

storage:
  trace:
    backend: local
    local:
      path: /tmp/tempo/traces

docker-compose:

otel-collector:
    image: otel/opentelemetry-collector-contrib:latest
    command: ["--config=/etc/otel-collector-config.yaml"]
    volumes:
      - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
    depends_on:
      - tempo
      
  tempo:
    image: grafana/tempo:latest
    command: ["-config.file=/etc/tempo.yaml"]
    volumes:
      - ./tempo-config.yaml:/etc/tempo.yaml
    ports:
      - "4318:4318"  # Exposing the new port (4318)
    
  grafana:
    image: grafana/grafana:latest
    ports:
      - "3000:3000"
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=admin
    depends_on:
      - tempo
2

It looks like otel collector doesn’t received any traces. You may check collector own metrics and check if receivers report any received spans.

3

i was able to update my docker-compose to this:

  otel-collector:
    image: otel/opentelemetry-collector-contrib:latest
    command: ["--config=/etc/otel-collector-config.yaml"]
    volumes:
      - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
    depends_on:
      - tempo
    ports:
      - "4317:4317"
    networks:
      - shuttle-network
  
  tempo:
    image: grafana/tempo:latest
    command: ["-config.file=/etc/tempo.yaml"]
    volumes:
      - ./tempo-config.yaml:/etc/tempo.yaml
      - ./tempo-data:/tmp/tempo  # Host directory for storing trace data
    ports:
      - "4319:4319"
    environment:
      - LOG_LEVEL=debug  # Set logging level to debug
    networks:
      - shuttle-network

the solution was to add ports section in otel-collector and also networks section in both. now i see in my otel collector container logs that im receiving spans. but its not exporting to my tempo container. i dont see any logs that my tempo container is receiveing spans. is there a config set up that im missing?

4

i read that otel collector is able to receive traces via otlp

5

i also upated my tempo.yaml to this:

but im unable to receive any traces from my otel collector. is this missing a config setup?

auth_enabled: false

server:
  http_listen_port: 3200

distributor:
  receivers:
    otlp:
      protocols:
        grpc:
          endpoint: 0.0.0.0:4319


ingester:
  trace_idle_period: 10s
  max_block_duration: 5m

compactor:
  compaction:
    block_retention: 1h

storage:
  trace:
    backend: local
    local:
      path: /tmp/tempo/traces