Connecting k6 to postgres DB

Grafana k6 Converters & Integrations
1

Hi, I’m trying to use https://github.com/imiric/xk6-sql to connect to a postgres database. I’m getting the following go error:

    ERRO[0001] panic: runtime error: invalid memory address or nil pointer dereference
goroutine 57 [running]:
runtime/debug.Stack(0xc000544808, 0xc000544828, 0xc0005447f8)
        runtime/debug/stack.go:24 +0x9f
github.com/loadimpact/k6/js.(*VU).runFn.func1(0xc00036b950, 0xc000545bc0)
        github.com/loadimpact/k6@v0.30.1-0.20210223102010-51790fc8df37/js/runner.go:705 +0x1ee
panic(0x1227480, 0x1f84790)
        runtime/panic.go:969 +0x1b9
github.com/dop251/goja.AssertFunction.func1.1(0xc000545ac0)
        github.com/dop251/goja@v0.0.0-20210216182323-60bc6ebb9fc1/runtime.go:2026 +0x85
panic(0x1227480, 0x1f84790)
        runtime/panic.go:969 +0x1b9
github.com/dop251/goja.(*vm).try.func1(0xc00057a000, 0x0, 0xc000545980, 0x0, 0x0, 0x0, 0xc000545a08)
        github.com/dop251/goja@v0.0.0-20210216182323-60bc6ebb9fc1/vm.go:406 +0x5d3
panic(0x1227480, 0x1f84790)
        runtime/panic.go:969 +0x1b9
database/sql.(*Rows).Columns(0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
        database/sql/sql.go:2935 +0x5c
github.com/imiric/xk6-sql.(*SQL).Query(0x21e29b0, 0xc002a0a1a0, 0xc002296000, 0x23, 0x0, 0x0, 0x0)
        github.com/imiric/xk6-sql@v0.0.0-20210112090239-90c72e1ef757/sql.go:49 +0x9b
reflect.callMethod(0xc002205c40, 0xc0005451c8, 0xc0005451b0)
        reflect/value.go:733 +0x1f6
reflect.methodValueCall(0xc002a0a1a0, 0xc002296000, 0x23, 0x0, 0x0, 0x0, 0xc001009bc8, 0xc000d0dae0, 0xc000545420, 0x4e02c7, ...)
        reflect/asm_amd64.s:35 +0x42
reflect.Value.call(0x11f1520, 0xc002205c40, 0x13, 0x13b4922, 0x4, 0xc001009b60, 0x2, 0x2, 0xc0004dd4a0, 0x41c9f8, ...)
        reflect/value.go:476 +0x8c7
reflect.Value.Call(0x11f1520, 0xc002205c40, 0x13, 0xc001009b60, 0x2, 0x2, 0xc001184738, 0x0, 0x0)
        reflect/value.go:337 +0xb9
github.com/dop251/goja.(*Runtime).wrapReflectFunc.func1(0x16154e0, 0xc000fe60f0, 0xc00057a3f0, 0x2, 0x7, 0xc0003ff9e0, 0x1)
        github.com/dop251/goja@v0.0.0-20210216182323-60bc6ebb9fc1/runtime.go:1668 +0x3c9
github.com/dop251/goja.(*vm)._nativeCall(0xc00057a000, 0xc000e88b40, 0x2)
        github.com/dop251/goja@v0.0.0-20210216182323-60bc6ebb9fc1/vm.go:1817 +0x2c2
github.com/dop251/goja.call.exec(0x2, 0xc00057a000)
        github.com/dop251/goja@v0.0.0-20210216182323-60bc6ebb9fc1/vm.go:1789 +0xaeb
github.com/dop251/goja.(*vm).run(0xc00057a000)
        github.com/dop251/goja@v0.0.0-20210216182323-60bc6ebb9fc1/vm.go:307 +0x99
github.com/dop251/goja.(*funcObject).call(0xc0029f5790, 0x1615da0, 0x21e2460, 0xc000d0da80, 0x1, 0x1, 0x0, 0x0, 0xc000644890, 0x41a0c5)
        github.com/dop251/goja@v0.0.0-20210216182323-60bc6ebb9fc1/func.go:161 +0x33a
github.com/dop251/goja.(*funcObject).Call(...)
        github.com/dop251/goja@v0.0.0-20210216182323-60bc6ebb9fc1/func.go:129
github.com/dop251/goja.AssertFunction.func1.2()
        github.com/dop251/goja@v0.0.0-20210216182323-60bc6ebb9fc1/runtime.go:2031 +0x96
github.com/dop251/goja.(*vm).try(0xc00057a000, 0xc000644a28, 0x0)
        github.com/dop251/goja@v0.0.0-20210216182323-60bc6ebb9fc1/vm.go:412 +0x163
github.com/dop251/goja.AssertFunction.func1(0x1615da0, 0x21e2460, 0xc000d0da80, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0)
        github.com/dop251/goja@v0.0.0-20210216182323-60bc6ebb9fc1/runtime.go:2030 +0x10b
github.com/loadimpact/k6/js.(*VU).runFn(0xc00036b950, 0x1604340, 0xc001009a40, 0x1, 0xc001564e40, 0xc000d0da80, 0x1, 0x1, 0x0, 0x0, ...)
        github.com/loadimpact/k6@v0.30.1-0.20210223102010-51790fc8df37/js/runner.go:710 +0x1b3
github.com/loadimpact/k6/js.(*ActiveVU).RunOnce(0xc0015ae800, 0x0, 0x0)
        github.com/loadimpact/k6@v0.30.1-0.20210223102010-51790fc8df37/js/runner.go:658 +0x226
github.com/loadimpact/k6/lib/executor.getIterationRunner.func1(0x1604300, 0xc00258e1e0, 0x15ea700, 0xc0015ae800, 0xc000d0d9d0)
        github.com/loadimpact/k6@v0.30.1-0.20210223102010-51790fc8df37/lib/executor/helpers.go:86 +0x5e
github.com/loadimpact/k6/lib/executor.PerVUIterations.Run.func3(0x15f6240, 0xc00036b950)
        github.com/loadimpact/k6@v0.30.1-0.20210223102010-51790fc8df37/lib/executor/per_vu_iterations.go:231 +0x237
created by github.com/loadimpact/k6/lib/executor.PerVUIterations.Run
        github.com/loadimpact/k6@v0.30.1-0.20210223102010-51790fc8df37/lib/executor/per_vu_iterations.go:244 +0xc9c

Goja stack:
file:///home/simon/Documents/git/test-sql-k6/script.js:23:302(69)
ERRO[0001] a panic occurred in VU code but was caught: runtime error: invalid memory address or nil pointer dereference  executor=per-vu-iterations scenario=default

I built k6 using the following commands

git clone https://github.com/k6io/xk6.git
cd xk6
CGO_ENABLED=1 go run ./cmd/xk6/main.go build master \
  --with github.com/imiric/xk6-sql

And here’s the script I’m running:

import sql from 'k6/x/sql';

const connStr = "postgres://postgres:abcd111@postgresql.url.net/employees?sslmode=verify-full"

const db = sql.open('postgres', connStr);

export function setup() {

    // Create data here.

}

export function teardown() {

  db.close();

}

export default function () {

//   db.exec(

//     "INSERT INTO keyvalues (key, value) VALUES('plugin-name', 'k6-plugin-sql');"

//   );

  let results = sql.query(db, 'SELECT * FROM departments LIMIT 10;');

  for (const row of results) {

    console.log(`key: ${row.key}, value: ${row.value}`);

  }

}

Can anyone help? It’s not possible to raise issues on the repo.

Thanks

2

Hi there and thanks for reporting this. I enabled the Issues on the repo so now these kinds of issues can be reported directly on GitHub.

This extension, as most xk6 extensions, isn’t well tested or maintained, so we missed this issue with Postgres. Sorry about that.

I pushed a fix to better handle errors, so if you rebuild it with xk6 now you’ll get better visibility of any errors instead of that nasty panic stack trace.

It’s difficult to say what’s the particular error in your case, but from my quick testing I was getting pq: SSL is not enabled on the server, so I had to pass sslmode=disable in the connection string, since I was running Postgres in a local container.

Another error running sql.query() could be pq: relation "departments" does not exist, so you’ll have to run it again and hopefully see a more helpful message this time.

Let us know here or if you have another issue, feel free to open a new one in the repo. Just keep in mind that this is not officially supported by the k6 team, and is more of a proof-of-concept project. :wink:

1 Like
3

How to build it under windows I’m getting command CGO_enabled=1 not found when running
CGO_ENABLED=1 go run ./cmd/xk6/main.go build master
–with GitHub - grafana/xk6-sql: k6 extension to load test RDBMSs (PostgreSQL, MySQL, MS SQL and SQLite3)

4

Use the commands here GitHub - grafana/xk6: Build k6 with extensions (you must have ‘go’ installed)

i.e.

go get -u github.com/k6io/xk6/cmd/xk6

xk6 build v0.30.0 --with github.com/imiric/xk6-sql

That will build the ‘k6’ executable (for k6 version 0.30.0) in the directory that you’re in.

Should work. I haven’t tried it.

1 Like
5

@Sergiu Besides having the Go toolchain installed, you also need a C compiler like gcc installed. I just tested on Windows 10 and this one worked fine: tdm-gcc. Make sure you enable “Add to PATH” in the installation options.

Then the way you specify environment variables is different on cmd.exe. This worked for me:

set GOOS=windows
set GOARCH=amd64
set CGO_ENABLED=1
xk6 build v0.30.0 --with github.com/imiric/xk6-sql
1 Like
6

OK, thank you, I was able to build it locally by using this commands

7

Hello!
I wanted to connect to the database, but when I run the script, it gives the following error:

image
image812×76 3.13 KB

How can I resolve this error?
I would be grateful for any help.

8

Hi @Sholpan ,

Given this error I would guess one of the following (in decreasing likelihood):

  1. You ran exactly the command above xk6 build v0.30.0 --with github.com/imiric/xk6-sql which unfortunately is now kind of broken as we did change some module paths and new versions of xk6 and modules (xk6-sql is updated) are not compatible with k6 pre v0.32.0 and vice versa. running xk6 build --with github.com/imiric/xk6-sql will fix it for you.
  2. You have an old version of xk6 in which case you can run go install github.com/k6io/xk6@latest
  3. You also could’ve not run the command or not then executed as ./k6 run .... but k6 run ... which would’ve not used the newly build k6.

Hope this helps you, and sorry for the inconvenience :frowning:

9

Hello!
I have a new version of k6 installed v32.0.

Do I need to install an old version к6 to use xk6?

10

Do I need to install an old version к6 to use xk6?

No.
Did you try running xk6 build --with github.com/imiric/xk6-sql ? (without the v0.30.0)

11

да, вышла следующая ошибка:

image
image1790×268 21.5 KB

maybe I’m installing it incorrectly:

  1. I installed go
  2. I downloaded the binaries
  3. then I ran the command in this folder: xк6 build --with GitHub - grafana/xk6-sql: k6 extension to load test RDBMSs (PostgreSQL, MySQL, MS SQL and SQLite3)
12

It seems like you can’t open https://proxy.golang.org/go.k6.io/k6/@v/list … can you try in a browser/using curl?

You might be behind a firewall or something that prevents you :(.

I think GOPROXY="direct" xk6 build --with github.com/imiric/xk6-sql should work for you

13

the browser opens, using curl no

I am on a corporate network

no, it didn’t work
thanks, I will continue to look for solutions

14

Hello!
if i work with a local directory , could I replace the path?

15

Can you elaborate on which path? if you are talking about the extension one you can do
xk6 build --with github.com/k6io/xk6-sql=. where . means the current directory but you can use relative or absolute paths.

You can also change the k6 path as in

# Build using a k6 fork repository from a local path. The version must be omitted
# and the path must be absolute.
$ XK6_K6_REPO="$PWD/../../k6" xk6 build \
    --with github.com/k6io/xk6-sql

both of which are shown in the xk6 examples