Run all tests scripts in parallel

[rakeshsukla53]

We use K6 to do our functional, performance and load testing. Currently we have 50-60 functional tests which are running daily, and we would love to run them in parallel if there is an efficient way.

There are some discussions around this topic I found here using docker which we are thinking about implementing, if there is no other solution. We will be using k6 to run our functional tests so the total number of cases will keep on increasing. Any ideas or suggestions for us to efficiently run all the tests quickly ?

How to execute multiple k6 scripts via single command in loadimpact/k6?

[nedyalko]

@rakeshsukla53

We use K6 to do our functional, performance and load testing. Currently we have 50-60 functional tests which are running daily, and we would love to run them in parallel if there is an efficient way.

With 50+ functional tests, having a separate scenario per test might not be the best option… I assume your functional tests just require a single VU each? If so, then you can encode your tests in an array, and then use the __VU (and potentially __ITER) execution context variables to partition the tests between your VUs and execute them in parallel. Something like this:

import { sleep } from 'k6';

const VUs = 6; // adjust this if you want more or less concurrency

var functionalTests = [];

function getTest(i) {
    return function () {
        return 'Test #' + i;
    }
}

for (var i = 0; i < 57; i++) {
    functionalTests.push(getTest(i));
}

const testsPerVU = Math.ceil(functionalTests.length / VUs);

export let options = {
    scenarios: {
        contacts: {
            executor: 'per-vu-iterations',
            vus: VUs,
            iterations: testsPerVU,
            maxDuration: '1h',
        },
    },
};


export default function () {
    var myTestNumber = (__VU - 1) * testsPerVU + __ITER
    if (myTestNumber >= functionalTests.length) {
        return; // last VU might have more iterations than needed
    }

    console.log(`VU ${__VU}, iter ${__ITER}, executed ${functionalTests[myTestNumber]()}`);
    sleep(1);
}

When we implement Improve execution information in scripts · Issue #1320 · grafana/k6 · GitHub and have a shared iterator, we’d be able to do something like this even more cleanly, but this should work until then, with just a few empty iterations that don’t run tests.

[rakeshsukla53]

Thank you @ned. I do want to clarify one thing when I said 50-60 functional tests that also means 50-60 separate test files for us. Currently they all run serially using a common default function. Is your answer still going to work in that case ?

[nedyalko]

You can keep them in separate tests files, so you can run them individually. But you can also have a single file that imports all of these separate files and executes their exported default functions concurrently, like I showed above. k6 has a require() function (that is very different than the one in NodeJS, see details in here, though you can use it with --compatibility-mode=extended as well), so you can have something like:

var functionalTests = [
    require("./functiona-test-001.js").default,
    require("./functiona-test-002.js").default,
    // ...
];

or do it programatically, and use the example I showed above to run all of the tests concurrently.

[rakeshsukla53]

Thank you @ned. All my test files also contains multiple groups so I am getting this error

ERRO[0015] GoError: Using group() in the init context is not supported at github.com/loadimpact/k6/js/common.Bind.func1 (native)

[nedyalko]

You shouldn’t run their default functions in the init context, only reference them. See:

require("./whatever.js").default

instead of

require("./whatever.js").default()

[rakeshsukla53]

Thank you @ned. This is a game changer for us. Thank you once again

[mrbaby]

I tried to follow the same, but somehow, it doesnt work.

import { sleep } from ‘k6’;

const VUs = 800; // adjust this if you want more or less concurrency

var functionalTests = [
require(“./LocationItemServices-clubItem-loadTest.js”).default,
require(“./locationItemServices-dcItem-loadTest.js”).default];

function getTest(i) {
return function () {
return ‘Test #’ + i;
}
}

for (var i = 0; i < 2; i++) {
functionalTests.push(getTest(i));
}

const testsPerVU = Math.ceil(functionalTests.length / VUs);

export let options = {
scenarios: {
contacts: {
executor: ‘per-vu-iterations’,
vus: VUs,
iterations: testsPerVU,
maxDuration: ‘30s’,
},
},
};

export default function () {
var myTestNumber = (__VU - 1) * testsPerVU + __ITER
if (myTestNumber >= functionalTests.length) {
return; // last VU might have more iterations than needed
}

console.log(`VU ${__VU}, iter ${__ITER}, executed ${functionalTests[myTestNumber]()}`);
sleep(1);

}

LocationItemServices-clubItem-loadTest file:
import grpc from ‘k6/net/grpc’;
import { sleep } from ‘k6’;
import { check } from ‘k6’;

const client = new grpc.Client();
client.load([‘definitions’], ‘item_apis.proto’);

let data = open(“./clubItemIds.csv”);
data = data.split(“\n”);

export let options = {
stages: [
{ duration: ‘2m’, target: 30 }, // simulate ramp-up of traffic from 1 to 30 users over 5 minutes.
{ duration: ‘5m’, target: 30 }, // stay at 30 users for 30 minutes
{ duration: ‘2m’, target: 0 }, // ramp-down to 0 users
],
};

export default () => {
if (__ITER == 0) {
client.connect(‘stg-score-item-services.walmart.com:80’, {
plaintext: true,
timeout:‘2s’
});
}

let ids = [];
let set = new Set();
for(let i = 0; i < 10; i++) {
    let id = getRandomId();
    while(set.has(id)) {
        id = getRandomId();
    }
    set.add(id);
    ids.push(`"${id}"`);
}
let queryMulti = `{

“itemIds”: [${ids}],
“locationId”:6612
}`;
const requestPayload = JSON.parse(queryMulti);
const response = client.invoke(‘com.sams.catalog.item.services.LocationItemService/GetLocationItems’, requestPayload);

check(response, {
    'status is OK': (r) => r && r.status === grpc.StatusOK,
});

if(response.status != grpc.StatusOK){
    console.log(JSON.stringify(response.body));
}
sleep(0.02);

};

function getRandomId() {
let randomNumber = Math.floor(Math.random() * 20);
return ${data[randomNumber]};
}

[prishu25]

Hi Ned can you share the full code

[prishu25]

@ned

import { sleep } from 'k6';

const VUs = 6; // adjust this if you want more or less concurrency

var functionalTests = [];

function getTest(i) {
    return function () {
        return 'Test #' + i;
    }
}

for (var i = 0; i < 57; i++) {
    functionalTests.push(getTest(i));
}

const testsPerVU = Math.ceil(functionalTests.length / VUs);

export let options = {
    scenarios: {
        contacts: {
            executor: 'per-vu-iterations',
            vus: VUs,
            iterations: testsPerVU,
            maxDuration: '1h',
        },
    },
};


export default function () {
    var myTestNumber = (__VU - 1) * testsPerVU + __ITER
    if (myTestNumber >= functionalTests.length) {
        return; // last VU might have more iterations than needed
    }

    console.log(`VU ${__VU}, iter ${__ITER}, executed ${functionalTests[myTestNumber]()}`);
    sleep(1);
}

I need Separate number of VU’s (so to achieve different TPS per scripts )and may be separate scenario for each Scripts.

[prishu25]

Hi ned,

This cmd i am using in bin/sh still k6 is not working in parallel. Is I am doing something wrong.

  set -x
  
  k6 run "/apps/k6perf/testapi/modifypickupcontact/modifyPickupContactFinal.js" >/dev/null &  
  pid1=$!
  
  k6 run "/apps/k6perf/testapi/cancelorder/cancelOrderMainFinal.js" >/dev/null &   
  pid2=$!
  
  k6 run "/apps/k6perf/testapi/additem/addPLItem.js" >/dev/null &
  pid3=$!
  
  k6 run "/apps/k6perf/testapi/priceunlock/priceUnlock.js" >/dev/null & 
  pid4=$!
  
  k6 run "/apps/k6perf/testapi/changepayment/changePayment.js" >/dev/null &      
  pid5=$!
  
  wait $pid1      
  wait $pid2      
  wait $pid3      
  wait $pid4      
  wait $pid5

  echo "All commands have finished."