Hoping that someone might take a look at the strange test behaviour we are observing
The script we are trying to test is a simple HTTP script with GET/POST requests. We are using the ramping-arrival-rate executor with the below workload model of our test
- Initial Load - 1 iteration per second.
- Target Load - 30 iterations per second.
- Stage 1 - Target 30, Duration 30 minutes
- Stage 2 - Target 30, Duration 60 minutes
- Stage 3 - Target 0, Duration 5 minutes
- Total Planned Duration - 1 hour 35 minutes 30 seconds
During the test, at multiple points (in all the stages), we have observed that k6 drops the load being sent to the system under test. For example, it scaled up to 2500 virtual users and then suddenly dropped to 300 virtual users. Now I would like to highlight that we observed a sudden surge in HTTP error responses from the application at the same time. The scripts are handled to use fail so in case of an error, the virtual user exits the current iteration as soon as it encounters an error.
My understanding is that when all the 2500 users are working simultaneously and all of a sudden there is a spike in errors causing let’s say 2200 virtual users to fail, all of these will immediately start the next iteration. So although there should be a drop in load due to the errors but not a huge drop as all the failed users should now restart the next iteration.
My questions are -
- When virtual users fail at different steps and abort the current iteration, how does the ramping-arrival-rate executor restarts all these users?
- Does it just drop all these users from the test and starts scaling again based on the load pattern?
- OR will it restart all the 2200 users to the next iteration at once?
- Documentation on fail() states that the virtual user will drop the current iteration but in the test, we have observed the user count drop from 2500 to 300.
Thank you so much in advance