I have two app versions. The old version sends some unnecessary API calls to backend from UI, they have been removed in the latest version. I ran a k6 test script against these two versions and got following results for each old and new versions. This script will execute a scenario where 100 VUs will try to run as many iterations as possible for 15 mins.
I can observe that no. of iterations has been increased in the latest versions, That means it has run the script more times in the new version than the old. so that according to my understanding, I conclude that page loading time has been increased in new version. Also, number of http_reqs to backend has been descreased (from 26k to 24k) in new version, therefore backend load burden also has been reduced. Can you help me to clarify whether my conclusions are valid or not?
Also, still I can see the http_req_duration is bit high in new version, then how can the new version execute more number of iterations?
This would be difficult to answer with certainty without seeing your script and being familiar with your system.
More script iterations could be because of several reasons: you’re doing less work per iteration (e.g. your script is taking a different path in the new version), or there were more shorter duration requests meaning that iterations are shorter. In general I would pay more attention to the http_req_duration metric to determine performance since iterations can be affected by anything else your script is doing besides making HTTP requests.
Your http_req_duration doesn’t look that much better in the new version, but the median value is lower, meaning there were more shorter-duration requests in the new version, which would explain the higher number of iterations. This would usually mean a higher number of requests as well, so I’m not clear why there were less in the new version.
You might also want to compare iteration_duration and add higher percentile stats with k6 run --summary-trend-stats="min,med,avg,max,p(99),p(99.9),p(99.99)" script.js so you can see the outliers. Judging by the p(90) value you also have more longer duration requests, which might explain the lower amount of requests overall.
Like I said, it’s difficult to arrive at a conclusion without knowing more details about your setup and running some more focused tests. Hope this helps.
