Prometheus remote write stale marker returns 400

I’m using K6 2.3.0 and use prometheus remote write extension to write data to on-prem Mimir.

I have enabled stale markers to prevent graphs in Grafana to continue when the test finished.

All works fine when I run tests up to 30 minutes. When I run a test of 1 hour, K6 fails with the following error:

time=“2026-10-13T08:19:12Z” level=error msg=“Stopping output 0 failed” component=output-manager error="marking time series as stale failed: got status code: 400 instead expected a 2xx successful status code"time=“2026-10-13T08:19:12Z” level=error msg=“Stopping output 0 failed” component=output-manager error=“marking time series as stale failed: got status code: 400 instead expected a 2xx successful status code”

The mimir logs show:

ts=2026-10-13T08:19:12.369935892Z caller=grpc_logging.go:76 level=warn method=/cortex.Ingester/Push duration=169.548µs msg=gRPC err=“user=ept: the sample has been rejected because another sample with a more recent timestamp has already been ingested and this sample is beyond the out-of-order time window of 5m (err-mimir-sample-timestamp-too-old). The affected sample has timestamp 2026-10-13T07:17:12.017Z and is from series k6_http_req_blocked_seconds{expected_response=\“true\”, group=\”::setup\“, k6_feature_native_histograms=\“true\”, method=\“GET\”, name=\“Zoek mappings\”, proto=\“HTTP/1.1\”, status=\“200\”, sut=\“WiremockOntwikkelPodman\”, testenv=\“ontwikkel\”, testid=\“20261013-091709\”, tls_version=\“tls1.3\”, url=\“Zoek mappings\”} (sampled 1/10)”

In the k6 test I am using a setup function to prepare a wiremock configuration that is used during the test. At the end of the test K6 seems to send a stale marker for the transactions in this setup function (mimir logs show group: setup). I’m not surprised that Mimir doesn’t accept a metric at 8:19 with a timestamp at 7:17.

Is this a bug, or am I doing something wrong?

I would expect K6 to send a stale marker for setup transactions when the setup function is finished. Although, if the same transaction is used later on in the test, this might cause issues. In that case, don’t send any stale markers for setup functions at all?