Hello
I have built a system that collects trace data from Python APP using OTEL using grafana Alloy, saves it in Tempo, and then monitors it using Grafana Drilldown.
As shown in the attached image, basic traces can be retrieved, but the problem is that whether I set the query condition to 1 hour or 3 hours, only 30 minutes of drilldown data is retrieved.
Can someone help me retrieve data normally regardless of the time set?
If you need more information, please ask and I will provide the information.
Thank you.
Tempo was installed using a helm chart, and the values of the helm chart are as follows.
tempo:
repository: <repository-path>
storage:
trace:
backend: s3
s3:
bucket: <bucket>
endpoint: <endpoint>
insecure: false
access_key: <access_key>
secret_key: <secret_key>
region:<region>
overrides:
# -- Default config values for all tenants, can be overridden by per-tenant overrides. If a tenant's specific overrides are not found in the `per_tenant_overrides` block, the values in this `default` block will be used. Configs inside this block should follow the new overrides indentation format
defaults:
metrics_generator:
processors:
# - service-graphs
# - span-metrics
- local-blocks
reportingEnabled: false
metricsGenerator:
# -- If true, enables Tempo's metrics generator (https://grafana.com/docs/tempo/next/metrics-generator/)
enabled: true
remoteWriteUrl: "http://prometheus-server.prometheus/api/v1/write"
queryFrontend:
search:
query_ingesters_until: 30m
query_backend_after: 30m
metrics:
query_backend_after: 30m
max_duration: 6h