Setting up self-managed grafana in my windows server and running Tempo services.
Open telemetry agent is running on my application server and sending the trace data to the local Tempo service.
Facing some issues when run the on-prem grafana & tempo.
- Getting the Gateway timeout error most of the time when explore the traces.
Sometime the traces are visualized but not all the time.
tempo.yaml:
server:
http_listen_port: 3200
#log_level: debugdistributor: # receive spans and forward them to the appropriate ingesters.
receivers:
otlp:
protocols:
http:
endpoint: 0.0.0.0:4318
#endpoint: localhost:4318 # No connection could be made because the target machine actively refused it.
grpc:
endpoint: 0.0.0.0:4317compactor:
compaction:
block_retention: 720h # configure total trace retention here. 30days.storage:
trace:
backend: local
wal:
path: C:\Program Files\GrafanaLabs\tempo\wal # where to store the head blocks while they are being appended to.
local:
path: C:\Program Files\GrafanaLabs\tempo\blocks # where to store the blocks.
- If configure endpoint as localhost:4317, getting below error & but no error when use endpoint with 0.0.0.0
2024-03-12T12:28:53.089Z warn zapgrpc/zapgrpc.go:195 [core] [Channel #1 SubChannel #2] grpc: addrConn.createTransport failed to connect to {Addr: “xx.xx.xx.xx:4317”, ServerName: “xx.xx.xx.xx:4317”, }. Err: connection error: desc = “transport: Error while dialing: dial tcp xx.xx.xx.xx:4317: connectex: No connection could be made because the target machine actively refused it.” {“grpc_log”: true}
