Websocket strange closing strategy

Grafana k6 OSS Support
1

i am doing websocket load testing using k6

k6 v0.39.0 (2022-07-05T10:50:14+0000/v0.39.0-0-g5904fd8, go1.18.3, linux/amd64)

what i want to do

  • open 4096 connections in 10s
  • keep all connection for 20s
  • close all connection in 10s

my simplified code

// setup k6 options
export const options = {
  scenarios: {
    load_testing: {
      executor: "ramping-vus",
      startVus: 0,
      stages: [
        // ramping-up
        { duration: "10s", target: 4096 },

        // keep
        { duration: "20s", target: 4096 },

        // it seems like that k6 won't close websocket automatically when vu ramping-down
        // so below line won't 'work' actually
        // { duration: "10s", target: 0 },
      ],

      // the gracefulStop does works, it will ramping-down all ws connections in "10s" !!!
      gracefulStop: '30s'
    },
  },
};

// websocket duration
// i don't want to close any websocket connection at ramping-up and keep stage
// so duration set to 10(ramping-up stage time) + 20(keep stage time) = 30
const websocketDuration = 30 * 1000;

// vu code
export default function () {
  ws.connect(url, function (socket) {

    // we need to close the websocket manually because the docs says
    // https://k6.io/docs/javascript-api/k6-ws/connect/
    // Calling connect will block the VU finalization until the WebSocket connection is closed. 
    // Instead of continuously looping the main function (export default function() { ... }) over an over
    // each VU will be halted listening to async events and executing their event handlers until the connection is closed.
    socket.setTimeout(() => {
      socket.close();
    }, websocketDuration);
  });
}

so here is the strange thing that i cannot find any doc related to it, gracefulStop will actually do the ramping-down work, and ramping-down time will equal to ramping-up time. i am very confused and not sure if it’s right.

2

Hello @mikcczhang ,
I would try adding a third step in your stages, going to 0 in 10 seconds, and adding a graceful ramp down…

        { duration: '10s', target: 0 },
  ],
  gracefulRampDown: '0s',

Hope that works :slight_smile:

3

Thank you for your reply.

I try your solution with following code but didn’t get what I expected. I am using k6’s test API so that you can easily run.

import ws from "k6/ws";

export const options = {
  scenarios: {
    load_testing: {
      executor: "ramping-vus",
      startVus: 0,
      stages: [
        { duration: "10s", target: 20 },
        { duration: "20s", target: 20 },
        { duration: "10s", target: 0 },
      ],
      gracefulRampdown: '0s'
    },
  },
};

const chatRoomName = 'publicRoom';
const websocketDuration = 30 * 1000;

// vu code
export default function () {
  const url = `wss://test-api.k6.io/ws/crocochat/${chatRoomName}/`;

  ws.connect(url, function (socket) {
    socket.on("open", function open() {
      console.log(`VU ${__VU}: connected`);
    });

    socket.setTimeout(() => {
      socket.close();
    }, websocketDuration);
  });
}

image
image1283Ă—390 41.4 KB

this result show that

  • it makes 29 sessions to the server, while expect 20
  • there are interrupted iterations

but if I change the code a little

  1. remove { duration: "10s", target: 0 },
  2. replace gracefulRampdown: '0s' with gracefulStop: '30s'

I can get the result what I expected

running (0m40.8s), 00/20 VUs, 20 complete and 0 interrupted iterations
…
ws_sessions…: 20 0.490745/s

5

Hi @mikcczhang, your original code seems fine. Adding one more stage in your case will let the VUs that started first to have time to finish 1 iteration and start over - starting a new one.

I would expect this was the original question? And you didn’t have some other problem?

if ramping down and graceful stop match the one that is shorter will be the one that will stop a VU. When you add one more stage (to target 0) graceful ramp down will start stopping VUs that are still running.

In your later example (with gracefulRampDown:0) at the 30s mark we start to go down from 20 to 0 over 10s so every 0.5s a VU will be stopped (forcefully). As k6 in practices stops the last started VU first that means that it shouldn’t have had time to finish an iteration (as it takes 30s with that timeout). So this is where the bunch of the interrupts come from. The 9 that finish are the first 9 and then they try to do another iteration as I said before.
(stopping first the VUs that have started last is an implementation detail and I would recommend nobody to depend on it :wink: )

Hope this answers you!

As general recommendation if you want to only do 1 iteration:

  • I would probably either not use stages, althought that means that all the opens will try to happen very fast(this might be less relevant in the case with 4096 as I would expect in that case that will still be true ;))
  • I would add an if the beginging to check if this is the first Iteration and if not to sleep for some amount of time - this will make certain that even if you get some tiem slightly wrong or k6 doesn’t stop a VU fast enough as it has too much work - you still won’t start one more iteration. It does mean that you might get an interrupted iteration or a few extra iterations.

I would also recommedn you to try xk6-websockets as that will let you write some async code and also will let you not run 4096 VUs just to open and close 4096 websocket connections. After all VUs are a lot more heavy.

If you do, please give some feedback as this extension will soon come into k6 core as experimental and hopefully in the future will be added in k6 core as stable.

1 Like