I am getting two issues when running k6 with a binary built to use the remote write extension for Prometheus. I am writing to a Thanos endpoint and am getting to warnings that I would like to fix but can’t find clear documentation on how to fix:
time="2022-04-11T19:31:28Z" level=error msg="Failed to store timeseries." error="server returned HTTP status 413 Request Entity Too Large: <html>"
and
time="2022-04-11T19:31:28Z" level=warning msg="Remote write took 3.974517546s while flush period is 1s. Some samples may be dropped."
is there any clear documentation on a.) whats happening here and b.) how to fix my implementation of k6 / thanos or prometheus to resolve these errors? Otherwise loving this tool so much!!
Both messages seem to be coming from the system under test, I guess Thanos in this case, or some proxy/lb before them (for the first one). Both seem like some limits that need to be raised, or your script changed to accommodate them - probably though sizing around what the real world use case will be.
So I would recommend asking through their channels.
It might also be worth opening an issue on the extension. While I have worked on it - I have done so in my capacity as k6 developer and understanding how to make the extension faster. I have little to no understanding of how to configure cortex, mimir, prometheus or thanos. And all the other people who worked on that project likely do not look at this forum.
