Suggestion on Approach: Multiple Scenarios and p(n) function inside checks

Grafana k6 OSS Support
1

@ned Hello , I would really appreciate your comments on our approach.

We are trying to use K6 to check performance but not in aspect of load.
Our use case is to run a request and verify that request respects the benchmarking.
We have more than 100 scenarios and it will increase

Currently we have 100 scenarios in one file and are running in parallel using per-vu-iterations.
We have define 100 scenarios in options as well.

Our main goal is to run individual request 5 times so we have kept options for each scenario as below.

export let options = {
    discardResponseBodies: false,
    scenarios: {
        GetApplets: {
            executor: 'per-vu-iterations',
            exec: 'getApplets',
            vus: 1,
            iterations: 5,
            maxDuration: '50s',
        }});

I also want to add a check for each scenario using p(n) function. Can we use p(n) inside check function? If not, is there a way we can put p(n) w.r.t each scenario ? (as p(n) differs for each scenario in our case)

example:

    check(response, {
        [description]: () => response.http_req_duration = ['p(80)<20000'] || fail("http_req_duration is null")
    });

P.S : Advance Example I referred last example were we can set for individual scenario inside option but couldn’t find anything w.r.t check function.

Your comments will be appreciated. Thank you @ned

2

No, you can’t use percentiles (p(n)) in a check(). You can use check() for for a single data point (e.g. response.timings.duration, see http.Response docs), while percentiles by definition concern multiple data points. The way to check percentiles in k6 is via thresholds: Thresholds

And when you have multiple scenarios, you can easily define a different threshold per scenario, see this advanced example. Something like this:

export let options = {
    scenarios: {
        GetApplets: {
            executor: 'per-vu-iterations',
            exec: 'getApplets',
            vus: 1,
            iterations: 5,
            maxDuration: '50s',
        },
        // ...
    },
    thresholds: {
        'http_req_duration{scenario:GetApplets}': ['p(80)<20000'],
        'http_req_duration{scenario:OtherScenarioName}': ['p(99)<300'],
        // ...
    },
};

And if you have 100 scenarios that are very similar, you might be better off generating the options.scenarios and options.thresholds programmatically before you export them to k6? :man_shrugging:

3

Thank you for your detail explanation.