K6 halting after a while of sending requests

Grafana k6 OSS Support
1

Hey all. I’m facing a pretty urgent issue I can’t seem to resolve. I’m experiencing k6 slowing down sending requests (I see rps decreasing in k6 stats), and eventually, requests seem to stop being sent altogether, for a period of more than 20 seconds, at which point I kill k6 (programatically). Has anyone experienced this before, and if so, do you have any recommendations for how to solve this issue? This happens with both a small number of VUs (4) and a larger number (around 80). My default function simply makes a request (with some custom logic), with a timeout of 5000 ms, and has a think time of 1 second at the end.

2

Can you share a bit more information? Which k6 version are you using, what are the other script options besides vus, etc?

Also, have you tried adding a console.log() statement in the code to see what’s happening in real time? Maybe the remote server starts to respond really slowly and VUs are stuck waiting for the requests to finish - have you tried reducing the timeout option for the HTTP requests?

3

Thank you for responding. I’m running v0.24.0. The only other script options are an environment variable to indicate the base URL to send requests to, and an output file to write JSON output to.

I haven’t tried putting console.log statements, but I’m looking at the logs of the server that should be receiving requests, and it is indeed not receiving anything. What kind of stuff should I be logging? I haven’t tried to reduce the timeout, but as far as I understand, the maximum amount of time before I get a response would be 5000 ms + 1000 ms think time, which is well below the timeout threshold of 20 seconds in my application, at which point I kill k6 (which is its child process).

I also suspected it has to do with VUs getting stuck and waiting for requests to finish. However, the timeout seems short enough for this not to be an issue.

Am I understanding correctly that the maximum amount of time k6 should take to log an http_req_duration event is 6 seconds? If I’m not, could you help me understand more clearly how this would work?

4

The only other script options are an environment variable to indicate the base URL to send requests to, and an output file to write JSON output to.

I was asking for the rest of the options, since originally you only mentioned 4 and 80 VUs. But you didn’t specify how you’ve configured the script execution length - do you use duration and/or iterations and/or stages? Asking, because there might be an issue like this: Stuck in infinite loop when specifying iterations and VUs · Issue #812 · grafana/k6 · GitHub

Regarding the request timeout, sorry, I missed the “with a timeout of 5000 ms” part of your original post. You are right, that should be sufficient to prevent requests from waiting on the remote server.

I’m not sure how to further help you diagnose the issue. You can try to run k6 in verbose mode (k6 run --verbose script.js) or you can use the console.log() statement as a very primitive debugging aid by having console.log(`VU ${__VU}, iteration ${__ITER}, ...`) statements in one or more places in the default function to see where it gets stuck.

5

I use the following function to configure my stages:

function initStages() {
  const USERS_PER_S = 3;
  const MAX_USERS = 100;
  const INTERVAL_S = 10;
  const LAST_INTERVAL_DURATION_MIN = 500;

  let users = 1;
  const stages = []
  
  while (users < MAX_USERS) {
    let stage = {
      duration: `${INTERVAL_S}s`,
      target: users,
    };
    stages.push(stage);
    users = Math.min(
      MAX_USERS,
      users + (USERS_PER_S * INTERVAL_S)
    );
  }

  stages.push({ 
    duration: `${INTERVAL_S}s`,
    target: MAX_USERS,
  });
  
  stages.push({
    duration: `${LAST_INTERVAL_DURATION_MIN}m`,
    target: MAX_USERS,
  });

  return stages;
}

This generates stages with a the last two stages being at 100 VUs for 10 seconds and 500 minutes, respectively. This is too long, so I actually killl k6 after a certain number of iterations. I don’t specify iterations anywhere in options though. That is done externally.

I also have a setupTimeout of 45s, and a few thresholds, which don’t seem to be failing.

6

initStages() seems fine to me. No idea what the issue could be, sorry. I tried to run a script with those stages and k6 0.24.0 and everything seemed fine. Did you notice anything strange when you use verbose mode and k6 gets stuck?