Send OpenTelemetry metrics to DataDog

Grafana k6
22

Thank you for your quick response, As per your inputs
I updated the container with -p 4318:4318 then I executed the below command but not work getting the same issue
Command:
K6_OTEL_HTTP_EXPORTER_INSECURE=true K6_OTEL_HTTP_EXPORTER_ENDPOINT=localhost:4318 K6_OTEL_METRIC_PREFIX=k6_ k6 run --tag test-id=Perf123 -o experimental-opentelemetry scripts/test.js

      /\      |‾‾| /‾‾/   /‾‾/        /\  /  \     |  |/  /   /  /        /  \/    \    |     (   /   ‾‾\     /          \   |  |\  \ |  (‾)  |   / __________ \  |__| \__\ \_____/ .io
 execution: local

script: scripts/test.js
output: opentelemetry (grpc, localhost:4317)
scenarios: (100.00%) 1 scenario, 5 max VUs, 10m30s max duration (incl. graceful stop): * default: 30 iterations shared among 5 VUs (maxDuration: 10m0s, gracefulStop: 30s)
2024/08/27 13:29:37 failed to upload metrics: context deadline exceeded: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: tls: first record does not look like a TLS handshake"2024/08/27 13:29:47 failed to upload metrics: context deadline exceeded: rpc error: code = Unavailable desc = connection error: desc = “transport: authentication handshake failed: tls: first record does not look like a TLS handshake”

Datadog agent container with -p 4318:4318 updated

DOCKER_CONTENT_TRUST=1 \
docker run --rm -d \
    --name datadog \
    -v /var/run/docker.sock:/var/run/docker.sock:ro \
    -v /proc/:/host/proc/:ro \
    -v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro \
    -e DD_SITE="datadoghq.com" \
    -e DD_API_KEY=<YOUR_DATADOG_API_KEY> \
    -e DD_DOGSTATSD_NON_LOCAL_TRAFFIC=1 \
    -p 4318:4318/udp \
    datadog/agent:latest
23

Why are you using udp in -p 4318:4318/udp? It isn’t required, you need to use tcp which is the default form so you don’t have to specify it as I posted in my previous examples.

24

it’s not allowing to install getting the below error

docker: Error response from daemon: driver failed programming external connectivity on testDrive datadog : Error starting userland proxy: listen tcp4 x.x.x.x:4318: bind: address already in use.

25

It sounds like you’re struggling with the Docker networking setup more than k6 and related. I suggest asking for support on Docker’s community/forum or StackOverflow to get better support.

Here, we mostly focus on k6/Grafana products.

26

I resolved that datadog issue and installed datadog agent successfully then I execute the below command and observed the different kind of issue

K6_OTEL_HTTP_EXPORTER_INSECURE=true K6_OTEL_HTTP_EXPORTER_ENDPOINT=localhost:4318 K6_OTEL_METRIC_PREFIX=k6_ k6 run --tag test-id=Perf123 -o experimental-opentelemetry scripts/test.js

/\ |‾‾| /‾‾/ /‾‾/ /\ / \ | |/ / / / / / \ | ( / ‾‾\ / \ | |\ \ | (‾) | / __________ \ |__| _\ ____/ .io
execution: local script: scripts/test.js output: opentelemetry (grpc, localhost:4317)
scenarios: (100.00%) 1 scenario, 5 max VUs, 10m30s max duration (incl. graceful stop): * default: 30 iterations shared among 5 VUs (maxDuration: 10m0s, gracefulStop: 30s)
2024/08/27 13:56:28 failed to upload metrics: context deadline exceeded: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:4317: connect: connection refused"2024/08/27 13:56:38 failed to upload metrics: context deadline exceeded: rpc error: code = Unavailable desc = connection error: desc = “transport: Error while dialing: dial tcp 127.0.0.1:4317: connect: connection refused”

27

I see where the issue might be. You’re missing this option K6_OTEL_EXPORTER_TYPE=http.

Can you try adding it to the last posted command, please?

28

I tried with your suggestions getting below error

2024/08/27 14:55:41 failed to upload metrics: Post “http://localhost:4318/v1/metrics”: read tcp 127.0.0.1:45544->127.0.0.1:4318: read: connection reset by peer

29

You should check the logs of your DataDog agent, and see why it closed the connection.

30

in the logs, I have seen this information after executing the command and continuously seeing the above issue

2024-08-28 06:19:38 UTC | PROCESS | INFO | (pkg/process/runner/runner.go:228 in logCheckDuration) | Finish container check #20 in 4.272398ms

2024-08-28 06:19:48 UTC | TRACE | INFO | (run.go:266 in Info) | No data received

2024-08-28 06:20:48 UTC | TRACE | INFO | (run.go:266 in Info) | No data received

2024-08-28 06:21:26 UTC | CORE | INFO | (pkg/serializer/serializer.go:418 in sendMetadata) | Sent metadata payload, size (raw/compressed): 2579/920 bytes.

2024-08-28 06:21:28 UTC | CORE | INFO | (pkg/serializer/serializer.go:442 in SendProcessesMetadata) | Sent processes metadata payload, size: 1410 bytes.

2024-08-28 06:21:58 UTC | TRACE | INFO | (run.go:266 in Info) | No data received

2024-08-28 06:22:58 UTC | TRACE | INFO | (run.go:266 in Info) | No data received

2024-08-28 06:22:58 UTC | PROCESS | INFO | (pkg/process/runner/runner.go:228 in logCheckDuration) | Finish container check #40 in 4.198326ms

2024-08-28 06:24:08 UTC | TRACE | INFO | (run.go:266 in Info) | No data received