OpenTelemetry Collector to Tempo

Grafana Tempo
,
1

Hi, i want to use Opentelemetry Collector to export my traces to Tempo (v2.6.1).

Opentelemetry tries to send (via HTTP) the endpoints to /v1/traces but tempo only accept /api/traces and /api/v2/traces.

does someone have a similar issue and maybe know how to handle it?

2

Enable/use otlp receiver:

3

thank you @jangaraj
I already enabled OTLP :thinking:

stream_over_http_enabled: true
server:
  http_listen_port: 3200
  grpc_listen_port: 3201
  log_level: info

query_frontend:
  search:
    duration_slo: 5s
    throughput_bytes_slo: 1.073741824e+09
  trace_by_id:
    duration_slo: 5s

distributor:
  receivers: 
    otlp:
      protocols:
        http:
        grpc:

compactor:
  compaction:
    block_retention: 1h # overall Tempo trace retention. set for demo purposes

metrics_generator:
  registry:
    external_labels:
      source: tempo
      cluster: docker-compose
  storage:
    path: /var/tempo/generator/wal
    remote_write:
      - url: http://prometheus:9090/api/v1/write
        send_exemplars: true
  traces_storage:
    path: /var/tempo/generator/traces

storage:
  trace:
    backend: local # backend configuration to use
    wal:
      path: /var/tempo/wal # where to store the wal locally
    local:
      path: /var/tempo/blocks

overrides:
  defaults:
    metrics_generator:
      processors: [service-graphs, span-metrics, local-blocks] # enables metrics generator
      generate_native_histograms: both

and here the available endpoints

GET /status/endpoints
+---------------------------------------------+---------------------------------------------------+
| NAME                                        | REGEX                                             |
+---------------------------------------------+---------------------------------------------------+
| /api/echo                                   | ^/api/echo$                                       |
| /api/metrics/query                          | ^/api/metrics/query$                              |
| /api/metrics/query_range                    | ^/api/metrics/query_range$                        |
| /api/metrics/summary                        | ^/api/metrics/summary$                            |
| /api/search                                 | ^/api/search$                                     |
| /api/search/tag/{tagName}/values            | ^/api/search/tag/(?P<v0>[^/]+)/values$            |
| /api/search/tags                            | ^/api/search/tags$                                |
| /api/status/buildinfo                       | ^/api/status/buildinfo$                           |
| /api/traces/{traceID}                       | ^/api/traces/(?P<v0>[^/]+)$                       |
| /api/v2/search/tag/{tagName}/values         | ^/api/v2/search/tag/(?P<v0>[^/]+)/values$         |
| /api/v2/search/tags                         | ^/api/v2/search/tags$                             |
| /api/v2/traces/{traceID}                    | ^/api/v2/traces/(?P<v0>[^/]+)$                    |
| /flush                                      | ^/flush$                                          |
| /generator/api/metrics                      | ^/generator/api/metrics$                          |
| /generator/api/metrics/query_range          | ^/generator/api/metrics/query_range$              |
| /ingester/ring                              | ^/ingester/ring$                                  |
| /memberlist                                 | ^/memberlist$                                     |
| /metrics-generator/ring                     | ^/metrics-generator/ring$                         |
| /querier/api/metrics/query_range            | ^/querier/api/metrics/query_range$                |
| /querier/api/metrics/summary                | ^/querier/api/metrics/summary$                    |
| /querier/api/search                         | ^/querier/api/search$                             |
| /querier/api/search/tag/{tagName}/values    | ^/querier/api/search/tag/(?P<v0>[^/]+)/values$    |
| /querier/api/search/tags                    | ^/querier/api/search/tags$                        |
| /querier/api/traces/{traceID}               | ^/querier/api/traces/(?P<v0>[^/]+)$               |
| /querier/api/v2/search/tag/{tagName}/values | ^/querier/api/v2/search/tag/(?P<v0>[^/]+)/values$ |
| /querier/api/v2/search/tags                 | ^/querier/api/v2/search/tags$                     |
| /querier/api/v2/traces/{traceID}            | ^/querier/api/v2/traces/(?P<v0>[^/]+)$            |
| /ready                                      | ^/ready$                                          |
| /shutdown                                   | ^/shutdown$                                       |
| /static/                                    | ^/static/                                         |
| /status                                     | ^/status$                                         |
| /status/overrides                           | ^/status/overrides$                               |
| /status/overrides/{tenant}                  | ^/status/overrides/(?P<v0>[^/]+)$                 |
| /status/usage-stats                         | ^/status/usage-stats$                             |
| /status/{endpoint}                          | ^/status/(?P<v0>[^/]+)$                           |
+---------------------------------------------+---------------------------------------------------+

API documentation: https://grafana.com/docs/tempo/latest/api_docs/
6

Can I Trust the /status/endpoints or are the the otlp endpoints not listed here if it is configured correctly?

7

OTLP GRPC is on the port 4317 and OTLP HTTP on the port 4318 by default (and you are using default otlp config).

1 Like