Grafana Agent auth handshake failed

Hello :slight_smile:

I’ve deployed a Grafana Agent for tracing and Grafana Tempo. Now I want to connect both with each other. The Agent is trying to export its (logged) data to Tempo Distributor on the jaeger grpc port (14250) and only returning me the following Errorlog: ts=2021-11-16T14:59:36Z level=error caller=exporterhelper/queued_retry.go:165 msg="Exporting failed. Try enabling retry_on_failure config option." component=traces traces_config=tempo-grafana-agent kind=exp │ │ orter name=otlp/0 error="failed to push trace data via OTLP exporter: rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: first record does not l │ │ ook like a TLS handshake\""

Did you ever happen to have this problem or know what the problem might be?

Best regards
Tom

It seems you are attempting to connect to a non-TLS endpoint using TLS. Perhaps try setting insecure=true on your write configuration?

1 Like

Thanks again, it works. So I have to adjust the port from 14250 to 55680.

If there are any ways to test if tracing works? Unfortunately I don’t know how I can test if Grafana agent got a trace and if the agent works properly with Tempo.

Thanks
Tom

The logs are good about reporting errors (as you’ve seen) if it is unable to forward traces to Tempo. I would also keep an eye on the following metrics emitted by the agent:

traces_receiver_accepted_spans
traces_exporter_sent_spans
traces_exporter_send_failed_spans
traces_receiver_refused_spans
1 Like