I have multiple micro services which sends traces to tempo. Each micro service has different SLO. I want to track the latency histogram based on each micro service.
How to configure multiple latency buckets for each microservice in metric generator in tempo
ex:
overrides:
metrics_generator:
processors:
- operation_aggregator
- service_aggregator
- span_metrics
span_metrics:
latency_histogram:
# Default buckets for latency in milliseconds
buckets: [10, 25, 50, 100, 200, 400, 800, 1600, 3200, 6400]
metrics:
# Service-specific latency histograms
- name: "http_request_duration_seconds"
description: "Latency of HTTP requests in seconds"
unit: "seconds"
tags:
- service.name
aggregations:
- histogram
# Service A
- name: "service_a_request_duration_seconds"
description: "Latency of requests in Service A"
unit: "seconds"
tags:
- service.name: "service_a"
aggregations:
- histogram
latency_histogram:
buckets: [5, 10, 20, 50, 100, 200, 500]
# Service B
- name: "service_b_request_duration_seconds"
description: "Latency of requests in Service B"
unit: "seconds"
tags:
- service.name: "service_b"
aggregations:
- histogram
latency_histogram:
buckets: [10, 25, 50, 100, 250, 500, 1000]
# Service C
- name: "service_c_request_duration_seconds"
description: "Latency of requests in Service C"
unit: "seconds"
tags:
- service.name: "service_c"
aggregations:
- histogram
latency_histogram:
buckets: [20, 40, 80, 160, 320, 640, 1280]