How to debug k6 testing script in Visual Studio Code?

When trying to launch Visual Studio Code in debugging mode, to inspect a k6 testing script, I get the following message:

Uncaught Error: Cannot find module ‘k6’.

I’m writing the test scripts in TypeScript and I’m using NodeJS + Webpack + Babel. The .js file I pass to k6 is the bundle file created by Webpack.

My launch.json file looks like this:

“version”: “0.2.0”,
“configurations”: [
{
“name”: “Launch Program”,
“type”: “node”,
“request”: “launch”,
“cwd”: “${workspaceFolder}”,
“protocol”: “inspector”,
“preLaunchTask”: “npm: build”,
“program”: “${workspaceFolder}/src/Main.ts”,
“outFiles”: [
“${workspaceFolder}/dist/Main.js”
],
“sourceMaps”: true
}
]

I’ve also added this to the tsconfig.json:

“sourceMap”: true

There is this import in my script:

import { Options } from ‘k6/options’;

Please, can anyone help? How can I configure it so that I can debug k6 testing scripts in Visual Studio Code?

1 Like

Hi @calderano,

I am afraid this is not possible :frowning: .

k6 is not nodejs based and also has no debugger of its own. So there is no way for visual studio code to debug k6 scripts. The error you see is because it tries to run it in nodejs and “k6” is not a module nodejs recognises.

Some thoughts on this:

  • k6 can get a debugger … I have no idea what this will entail, but it will probably be a lot of work and is likely to not be compatible with anything else, which will practically negate the majority of the benefits
  • we can reimplement the k6 modules for nodejs so you can debug them. There have been people who wanted to use their (request making) nodejs code to be run in k6, but if they use these modules in nodejs … it will also work in k6. The implementation won’t be exactly the same so there will be subtle differences which means that if you are debugging something complex you might get different behaviour. Also this will be a lot of work.

Are you doing something complex enough that console.log won’t help you? As this is the only "debugger’ currently and given my reasoning above, it is unlikely to change … soon :frowning: .

1 Like

Hi @mstoykov,

I was afraid of that but I was hoping there would be a way around it.
I understand. Thank you very much for the explanation.
We have just started a project using k6 and I was doing a ‘research task about it’. I believe console.log will help us.

Thank you once again!

I found it helpful to run k6 via HTTP proxy such as https://mitmproxy.org/. Also if you need to debug your own code and you factor it into a separate module, then you can use the usual node debugger tools.

3 Likes

is likely to not be compatible with anything else, which will practically negate the majority of the benefits

I think that integration of such a debugger with the IDE could be done with the comparable efforts.
Anyway, having a debugger for a tool with scripting capabilities is a big advantage for this tool.

1 Like

we can reimplement the k6 modules for nodejs so you can debug them.

That sounds like a good and straight forward solution to the problem. Is there a technical reason (e.g. performance) for the built-in modules not written in JS at the first place? Is it realistic to expect mirroring functionality to be implemented as an NPM module and supported alongside with the main product?

No, or at least the k6 core dev team doesn’t plan to do it. It’s quite the significant amount of work, and even if it’s done once, having to maintain 2 separate implementations of the same API will have huge ongoing maintenance costs.

The reasons the k6 core is written in Go are many, but the standard Go HTTP library and better multi-core performance are two big ones. Every VU in k6 is a separate and (mostly) independent JavaScript runtime.

Thanks for the prompt response! I didn’t mean to reimplement whole core and functionality in NodeJS instead or alongside Go implementation, you guys did a great job explaining the reasoning for using Go instead of NodeJS in the documentation. I apologize for not being clear.

What I mean was to keep core in Go, and either to reimplement the built-in module only in JS, or to have the built-in module JS implementation alongside with the implementation within the core. The JS implementation could be as simple as possible, even dummy stubs with the compatible signatures would be a great step froward (according to k6 JavaScript API it could be just few tens of simple functions).

That would let run a single VU in NodeJS to run the script using the NodeJS debugger. Actual load runs would still be via k6 core written in Go.

1 Like

We now have a significant amount of code in our k6 repo and we have come to a point where the lack of debugging is hurting us. I’ll investigate how much effort this actually is – seems reasonable for what we need and I suspect others as well.

1 Like

Hey!

Gist

There is a POC debugger for k6 that is not merged into Goja yet.

Details

A while ago I started experimenting with Goja, the underlying JS engine of k6 and DAP, LSP and LSIF on VS Code and I was convinced back then that the only way to debug k6 code is to implement a debugger in Goja, which was exactly what I did, although with its own limits and shortcomings.

I created a CLI frontend application, Goja Debugger, to demonstrate the capabilities of the new debugger. This new debugger has the basic functionalities needed for the Goja to be able to debug a JS script, as mentioned here. Also, there has been lots of discussions around it and I also filed a PR for review. You can also have a look at the tests @mstoykov and I wrote for the debugger implementation in Goja.

@mstoykov helped improve the code a lot after my initial POC and also created a POC debugger frontend with 1 VU inside k6 to show how the debugger works, which you can compile and test.

There is still road ahead for the debugger to be improved and possibly merged in Goja. Finally the DAP needs to be implemented as another frontend for the debugger. I created a stub/skeleton for the DAP server here.

Aside from @mstoykov and I, a lot of people helped make this possible, notably Niels Widger for testing and suggestions and recently Zhongxian Pan who contributed code and filed lots of issues and PRs. So, if you are interested in the debugger, please help test and improve/develop it. I’d be happy to help and review the PRs and answer the issues.

Good luck! :slight_smile:

3 Likes

There has to be a way to solve this. This is the #1 problem with k6.

There is a POC debugger for k6 that is not merged into Goja yet.

What ever became of this project? I see that the repo in GitHub was archived. Was it just abandoned? Combined into another project?

Hey @cfield,

The project is archived, because we are not going to work on it anymore AFAIK, aka. abandoned for now. But if there’s a contributor, I’d be happy to help review it. There are other issues as well, for example, you can’t debug your script in a distributed manner, hence you default to 1 VU for debugging, which might not be what you want. Otherwise your JS linter can probably catch a lot of issues. You can use the @types/k6 to further lint and validate your code.

2 Likes

I frequently debug my k6 scripts in node using VS code, I would find K6 unusable for my purposes without it and fall back onto something with better debugging functionality.

The method I use is to create a few polyfill functions and use the “require” syntax to determine which modules to load at runtime.

The things to keep in mind are that this method doesn’t support the k6 wrapper for load generation so the user model with which transactions are fed in is arbitrary. There are some differences in behavior between goja and node that you can come across too.

Its not perfect but gets the job done for testing functionality.

Lemme know if you want more information

3 Likes

This sounds interesting, as K6 still lacks convenient debugging. I would love to hear some details or see an example. Thank you!

2 Likes

Hey,

Sorry for late reply! Had trouble with my Grafana account so didn’t notice any replies.

Going into a bit of detail around the concepts:

For k6 we write javascript to invoke go sub routines
For nodejs we write javascript and invoke node sub routines

Effectively the only difference between both runtimes is therefore how we invoke subroutines and how our code is invoked by the runtime. The code in between is interopable.

Therefore if we sacrifice some of the invocation logic in k6 and replace the subroutines we invoke with node equivalents we can achieve a mostly consistent debug runtime. Whilst running in node we lose things like grafana integration and the execution model of k6 so this approach is predominately for debugging or low throughput testing. Importantly we don’t lose k6 functionality when running in k6

With the above in mind what I did in my codebase is as follows:

- Have a common entry point between k6 and node
This is where we export options and scenarios for k6 as normal
At the end of this file i have a dirty try catch block which references a protected variable exclusive to nodejs, in my case “process”
From here if process is defined (ie. the try block succeeds) my next operation is to invoke the function that we pass in on k6’s scenario. In pseudocode something like this:

try {process.env.scenario ? main(process.env.scenario) : null } catch(err){}

This gets us to the point where we can interop the entrypoint of our nodejs and k6 apps. Next obvious question is what about all the subroutines?

I create a file called k6PolyFill (for lack of a better name) and pop in a bunch of functions or singletons to interop between k6 native functions and node equivalents. The important thing to note here is that the functions we define in here should have the same invocation parameters and return data as the k6 equivalents for max capability. The functions I’ve interoped for my usage are:

  • sleep
  • SharedArray
  • Trend
  • Counter
  • exec
  • check
  • open

This file will conditionally export the node functions we’ve created OR alias the k6 functions depending on whether the env is node or k6 (as defined by the psuedo code earlier). You’ll need to use module.exports as an FYI since the ECMA import/export syntax doesn’t support conditional export

Special mention goes to:

  • Group
    At the time I wrote the codebase, group could not be awaited in k6 and made compatibility super ugly to deal with. In exchange I’ve defined a trigger function which computes the probability of a trigger based on the scenarios specified throughput

  • HTTP
    For this one I use nodejs HTTP/HTTPS libs and the native HTTP function for k6. There are a few flags that need to be added to the node calls to make it behave more similarly to k6. In addition I put in a whole bunch of standardised logic before exporting this one out for logs that can be forwarded to log ingestion engines and a bunch of standardised assertions

From here if you refer to only functions in polyfil instead of k6 native directly you can easily interop between nodejs and k6 as your runtime.

In all honesty, I have no idea how I would maintain the k6 scenarios that I have (which consist of thousands of API’s often in complex sequences) without the ability to debug with line breaks in an IDE

When i started and ran using traces natively in k6 with just a segment of the tests we have now, it was becoming impossible to maintain

Feel free to let me know if you guys want more details or code