Http only: retrieve multiple Set-Cookie-s from response

I’ve invested quite some time into writing a script suite for a scale test. I’m following industry standard procedure and went with a hybrid solution. The headless-browser scenario works fine and I’m struggling with the HTTP only one.

The user flow starts with a login from which I have to retrieve token and sessionID as well. Checking login on Chrome dev-tools, I can see that the POST login request returns 302 status and also 2 Set-Cookie objects in the response header.


I think, I’ve tried everything. I can only get the token back.

Here is the function for the login (resides in a separate file and imported in the main script):

export function httpApiLogin(httpClient, userFlowId, username, password) {
  let logPrefixLogin = getLogPrefix(userFlowId, __ITER, __VU, "Login API");

  const url = `${URL_BASE}${URL_PATH_LOGIN}`;
  const payload = JSON.stringify({
    username: username,
    password: password,
  });
  let params = {};

  // Landing page, let's get the arccsrftoken
  try {
    const res = httpClient.get(url);

    // Check if 'arccsrftoken' is set
    if (res.cookies['arccsrftoken']) {
        const csrfToken = res.cookies['arccsrftoken'][0].value;
        // console.debug(`arccsrftoken: ${csrfToken}`);
        params = {
            headers: {
                'Content-Type': 'application/json',
                'Cookie': `arccsrftoken=${csrfToken}`,
                'X-CSRFToken': csrfToken
            },
            redirects: 0
          };
    } else {
        console.error(`${logPrefixLogin} arccsrftoken not found in cookies`);
    }
  } catch (e) {
    console.error(`${logPrefixLogin} Error fetching arccsrftoken: ${e}`);
  }

  try {
    const res = httpClient.post(url, payload, params);
    // no sessionID in it only token
    //let jar = http.cookieJar();
    //let cookies = jar.cookiesForURL(res.url);    
    //console.log("CC::"+JSON.stringify(cookies));
    //console.log(`LOGIN COOKIES ${JSON.stringify(cookies)}`);
    const success = check(res, {
        'Login successful': (r) => r.status === 200,
    });
    console.log(`${logPrefixLogin} request status: ${res.status}`);

    console.log(`${logPrefixLogin} COOKIES: - - - - - - - - `);
    // Access all cookies from the response
    for (const name in res.cookies) {
      if (res.cookies.hasOwnProperty(name)) {
        // Each cookie name can have multiple values in an array
        res.cookies[name].forEach(cookie => {
          console.log(`Cookie: ${name}=${cookie.value}`);
          // Access other cookie properties if needed
          console.log(`Domain: ${cookie.domain}, Path: ${cookie.path}`);
        });
      }
    }

    console.log(`${logPrefixLogin} RAW COOKIES: - - - - - - - - `);
    // Get all raw set-cookie headers (returns an array of values)
    const setCookieHeaders = res.headers['Set-Cookie'];
    
    // Process each set-cookie header
    if (setCookieHeaders) {
      if (Array.isArray(setCookieHeaders)) {
        setCookieHeaders.forEach(cookieHeader => {
          console.log(`Set-Cookie header: ${cookieHeader}`);
        });
      } else {
        console.log(`Set-Cookie header: ${setCookieHeaders}`);
      }
    }

    console.log(`${logPrefixLogin} COOKIE JAR: - - - - - - - - `);
    // Get the VU cookie jar
    const jar = httpClient.cookieJar();
    
    // Get all cookies for the URL
    const cookies = jar.cookiesForURL(res.url);
    
    // Log all cookies
    for (const name in cookies) {
      console.log(`Cookie in jar: ${name}=${cookies[name][0]}`);
    }

    return {
        success: success,
        response: res
    };
  } catch (e) {
    console.error(`${logPrefixLogin} Error logging in: ${e}`);

    return { success: false };
  }
}

And this is how it is called from the main script:

export default function() {
  // User flow id generation for better logging
  const userFlowId = getUserFlowId();

  const randomUser = users[Math.floor(Math.random() * users.length)];

  let requestStart = new Date().getTime();
  const responseHttpApiLogin = httpApiLogin(http, userFlowId, randomUser.username, randomUser.password);

...
}

Could it be that my problem is similar to this? Get content of httponly cookie in k6-browser response Although, this one seems to be resolved.

I can’t imagine a tool like this doesn’t support the retrieval of all Set-Cookie objects. I must be missing something here. Any help appreciated. Thanks!

The problem could be with response I get at the first place and probably not how k6 handles it. Investigation WIP.

Hi @bgyomorei , welcome to the community forum!

Just to confirm you are not using browser at all and you do not have problem with it?

You do mention and link to a post about it, but if you are not using it, it doesn’t really matter.

By the info you have given, my expectation is that you are seeing this problem because you are getting redirected and you are trying to get info from the second response. But the second response is not the one that gets the Set-Cookie headers.

I would usually recommend using the CookieJar.cookiesForURL after you do the request. I see that you do that, and my guess is that maybe the after redirect is for a URL that the cookie isn’t set? Maybe try wiht the request URL ?

In general if this just works in a browser, it should just work here without you need to do anything as the cookies gets saved and propagated accordingly. But I see you set an additional header so maybe that is the reason you need this?

It also will be beneficial to have the output for your run and you can use either --http-debug or ssl keylogger setup with something like wireshark - unfortunately I still haven’t ended up writing proper documentation on that :frowning: