I am running my test scripts on my app which is also connected to new relic APM, well I can inspect the APM dashboard after every run and compare it with K6 report and they never align and it seems like K6 report very high numbers when it comes to response time.
My expectation is for APM transaction response time report in percentile to align with that of k6 metrics but the different is very huge. K6 says that at 95th percentile the http_req_time is 9.8s, meaning
Hi @kasongoyo, welcome to the forum!
Looking at the graph I see that you have the p95 at between 7.5 kms and 10kms or between 7.5 seconds and 10 seconds which probably on average is somewhere around 9.08seconds. So everything seems okay. Looking at the average it seems to be in line with what k6 reports as well. I think you got confused because the “kilo milliseconds” that New Relic is reporting instead of the just seconds that k6 does.
I would expect new relics numbers to always be slightly smaller as they don’t include the time from k6 to your application and back, but that should be negligible when both are close together.
Hope this helps you!
Thanks @mstoykov for the prompt reply. I could say it happened coincidentally that those two data align, but it doesn’t for most of the time. For example see below
As you can see the difference is huge and really.
I would expect, given the size of the http_req_waiting, which is how long k6 waited for the server to return something after it send the request (also known as time to first byte - TTFB), that in that test k6 is running on a different continent with a congested network or something (although your kb/s is also really small) and it will need to be very far apart for a 2s waiting time 
.
A different theory will be that you are not measuring with New Relic early enough:
ASCII graph of a request (very simplified):
k6 → k6’s network stack → wire/some routers → SUT’s network stack → your application
k6 measures the whole thing from the start to the end. New relic measures somewhere in your application which also has probably many parts. If you have for example a limiter somewhere that will only let through 10 requests but you measure with New Relic after the limiter has been passed, New Relic won’t measure the time a request spends in a limiter. This might not even be in your application but in your:
You can try to make a curl request like the ones that are slow in k6 (preferably from the same machine k6 is running from at the same time as running a k6 test) and see how long it takes.
Given that k6 has not shown anything like that before and you are comparing something measuring the whole time for the request compared to some part of what happens in your application, I would expect that you just haven’t measured correctly the second part. And again … k6 should always show bigger numbers as it includes more.
Hope this helps, and if you have any more info that points to k6 being at fault, please share it, especially if you can make a reproducible demo which the k6 team can run. As well as any other questions 
