Hi,
I’m running a test consisting of multiple scenarios and I have these thresholds set, independent per scenario.
When a scenario fails, it causes all of the tests to stop. Is there a way to only stop this specific scenario and continue with the others?
Which thresholds?
Do any of the thresholds have abortOnFail: true ?
Yes they have abortOnFail: true. Mostly httpReqDuration thresholds.
AFAIK that is a test-wide setting, not just for a scenario. Try setting the value to false or removing the property to verify. There is also a delayAbortEval property which will delay the evaluation of the threshold.
Everyone’s requirements are different. For httpReqDuration thresholds I have not been aborting at all, but have multiple tiers of test fail criteria. This way the test has collected full metrics for later analysis. I have abortOnFail: true for two thresholds: one related to httpReqFailed and one related to checks. In both cases, I allow a relatively trivial % to fail an initial threshold but not stop the test (will still end up as a Fail), and both have a lower threshold where it’s heading downhill fast, no sense in continuing so stop the test immediately.
I believe the only other workarounds for your situation is to run multiple separate scripts in parallel, or use modules.
1 Like
So there isnt a setting that only stops that specific scenario? Thats a shame. Okay in that case probably gonna have to turn every scenario into a separate script.
Support can confirm but it appears that this request is already being tracked under #1833, see this thread.