Remote write prometheus

I am trying to write k6 data to Amazon Prometheus but getting 403 responses -
level=error msg=“Failed to send the time series data to the endpoint” error=“got status code: 403 instead expected a 2xx successful status code” output=“Prometheus remote write”
I have added a policy to the role assigned to my user and tried running with ec2 as well but still having the same error.

Hi @neerajofficial0312

Welcome to the community forum :wave:

I am unfamiliar with Amazon Prometheus, though since 403 (Forbidden) is the usual error when trying to access a resource that you’re not allowed to access, I would indeed look into user access, as you have been doing. I would first try to figure out where the issue is, on the Amazon side, or k6 side.

What authentication options do you have in Amazon Prometheus, can you use HTTP Basic authentication, or do you need to pass a bearer token, etc.?

If you haven’t already, start by checking with a curl command if you can access the write point, and with what authentication types. E.g. for basic HTTP something similar to:

curl -X POST -H 'Authorization: Basic <base64_encoded_user_credentials>' https://<prometheus_hostname_url>/api/v1/write

Or with a bearer token:

curl -X POST -H "Authorization: Bearer <bearer_token>" https://<prometheus_hostname_url>/api/v1/write

If you get a 403 with curl, you would first need to investigate Amazon’s configuration. If curl works well, then you’ll need to try to configure the right options so that k6 uses the same authentication mechanism: Prometheus remote write | Grafana k6 documentation

I hope this helps :bowing_woman:

Cheers!

1 Like

Hi @eyeveebee ,

I am trying to run the k6 script from codebuild and my codebuild service role has the policy to remote write in Amazon Prometheus but still it throws 403.
So even after assigning the policy I need authentication at runtime?