Getting 401 (Unauthorized); transport: received unexpected content-type application/json when shipping traces using Grafana Agent

Grafana Tempo
1

Hi,

I was following this guide below to ship traces from k8s to grafana cloud:

But I keep on getting this error from grafana-agent-traces pod:

│ ts=2023-02-11T13:10:28Z level=error caller=exporterhelper/queued_retry.go:149 msg="Exporting failed. Try enabling retry_on_failure config option to retry on retryable errors" │
│  component=traces traces_config=default kind=exporter data_type=traces name=otlp/0 error="Permanent error: rpc error: code = Unauthenticated desc = unexpected HTTP status cod │
│ e received from server: 401 (Unauthorized); transport: received unexpected content-type \"application/json\""

Thanks.

2

I’m also facing same issue. @lordpangan did you find any solution?

3

Hi! Can you check that you’ve correctly configured basic auth?

4

I would say that Getting 401 (Unauthorized); transport: received unexpected content-type application/json indicates wrong Realms and/or Scopes configuration of used Access policy.

5

I had the same problem, to solve that is necessary to set the name as default in blocks receiver and batch, i don’t know why they send header with this information and consequently cause troubles.

otelcol.receiver.otlp “default” {
grpc {
endpoint = “0.0.0.0:4317”
}

http {
	endpoint = "0.0.0.0:4318"
}

output {
	traces  = [otelcol.processor.batch.default.input]
}

}

otelcol.processor.batch “default” {
output {
traces = [otelcol.exporter.otlp.grafana_cloud_tempo.input]
}
}

6

Does anyone find the solution?