How to properly test and run multiple browser separate tests in a single scenario?

Grafana k6 k6-browser
1

Hi there,

I am using and enjoying k6 (browser testing).

However, I am experiencing problematic behavior when executing 5 different tests in a single scenario in browser-level testing.

How could I actually execute them to correctly measure a site’s performance?

This is the scenario I have:

// Separate scenarios for separate testing needs
let scenarios = {
  tests: {
    executor: 'shared-iterations',
    vus: 1,
    iterations: 1,
    maxDuration: '5m',
    exec: 'tests',
  }
};

// All the tests ran for pull requests
export function tests() {
  test1();
  test2();
  test3();
  test4();
  test5();
}

I have all 5 tests created in a separate test file, so I created also one scenarios.js with the above codes (also including relevant thresholds, imports etc)

When I then execute the scenarios.js, I see 5 chrome browsers opened and executed one by one, since I’m using shared-iterations. If I use per-vus-iterations, it will try to execute at once, which is also not good for performance results.

Is this good way and the only way to use shared-iterations to execute them all? Are there any other methods (proper one) to execute the tests one by one, so I have correct results at the end?

I hope you got my point, please let me know if any questions.

2

Hi @moonbridge,

To understand the issue clearly, do you want to first execute test1 and test2 and so on? Or do you want to execute them in parallel?

Thanks.

1 Like
3

Hi @inanc
thanks for the response! Yes, I would like to execute them one by one when running scenarios.js. I believe it is the right approach to test browser-level tests since executing them all at once would be stressful for the server and the results might be wrong (too slow for the site).

4

Hi @moonbridge,

Can you give us a bit more context as to what exactly you are trying to test? We understand that it’s a website, but what are you trying to assert when the tests run?

Scenarios are a good way to emulate/model the traffic and see how your web services (or website) handles that load. shared-iterations will share the total number of iterations between all the VUs. So if you had 10 VUs and 100 iterations then each VU (which run in parallel) will run the test until a total of 100 iterations of the test completes.

Here are a couple of ways in which you could run the tests sequentially:

  1. Now that k6 supports async and await keywords, you could make each of the tests async and in the main tests function await on each of them. e.g.
    export async function tests() {
  await test1();
  await test2();
  await test3();
  await test4();
  await test5();
}

async function test1() {
  ... // test code
}

async function test2() {
  ... // test code
}

... // Other tests
  2. Check out this post on running tests sequentially using scenarios.

Cheers,
Ankur

1 Like
5

That actually solved the problem, thanks a lot! Now with added async and await to my functions (tests) it is running one by one.

1 Like
6

Hi Ankur, can you suggest the same for parallel. Thanks

7

Hi @aakash.gupta,

This feels more like a general JS question, which i’m sure you will be able to get an answer for elsewhere, but here’s a quick solution:

export async function tests() {
  await Promise.all([
    test1(),
    test2(),
    test3(),
    test4(),
    test5(),
  ]);
}

async function test1() {
  ... // test code
}

async function test2() {
  ... // test code
}

... // Other tests

Cheers,
Ankur

8

Sorry but let me Elaborate on my requirement. I have multiple test files and they all have different setups, some have more than 1 scenarios. How do I run them in parallel?

9

Hi @aakash.gupta,

Take a look at this thread which i think will answer your question.

Cheers,
Ankur