Go test fails on release tag

Grafana k6 OSS Support
1

Hi,
I’m wanting to try to fix my own filed k6 issue (#1804). I’m a hard greenhorn at Go but did golang’s tour so am kind of capable to read code. I really want this bug fixed and we all gotta start somewhere, right?

Right now I’m trying to set a baseline before I try out code changes locally by running go test ./... from the k6 root directory on a release tagged commit, say v0.33.0. However, I’m getting different FAILs at different places, depending on the platform, macOS or Win10.
My setup:
I installed go 1.16.3 on a macOS 11.5.1 via homebrew and go 1.16.7 on Win10 via golang’s own installer. Both systems are the same network.
On both machines, go get go.k6.io/k6 wasn’t setting up a GOPATH/src directory, as described on your Contributing page, so I’m working with the codebase cloned from github.
The go test FAILs:
On macOS:
a dozen

--- FAIL: TestDNSResolver (0.21s)
    --- FAIL: TestDNSResolver/cache (0.00s)
        --- FAIL: TestDNSResolver/cache/1000 (6.40s)
            local_test.go:1077: 
                        Error Trace:    local_test.go:1077
                        Error:          Error message not equal:
                                        expected: "dial tcp 127.0.0.254:52659: connect: connection refused"
                                        actual  : "request timeout"
                        Test:           TestDNSResolver/cache/1000
...
FAIL
FAIL    go.k6.io/k6/core/local  12.444s

On Win10:
this one (filepath redacted at “\…\”); german text because german Windows;

--- FAIL: TestClient (0.09s)
    --- FAIL: TestClient/LoadNotFound (0.00s)
        client_test.go:137:
                Error Trace:    client_test.go:137
                Error:          "./does_not_exist.proto: open C:\\Users\\...\\k6\\js\\modules\\k6\\grpc\\does_not_exist.proto: Das System kann die angegebene Datei nicht finden. at go.k6.io/k6/js/common.Bind.func1 (native)" does not contain "no such file or directory"
                Test:           TestClient/LoadNotFound
FAIL
FAIL    go.k6.io/k6/js/modules/k6/grpc  0.559s

a long string of these

time="2021-08-10T21:22:14+02:00" level=warning msg="Request Failed" error="error decompressing response body (gzip: invalid header)"
time="2021-08-10T21:22:14+02:00" level=info msg="Request:\nPOST /digest-auth-with-post/auth/testuser/testpwd HTTP/1.1\nHost: 127.0.0.1:58982\nUser-Agent: TestUserAgent\nContent-Length: 17\nAuthorization: Basic dGVzdHVzZXI6dGVzdHB3ZA==\nAccept-Encoding: gzip\n\nsuper secret body\n" group= iter=0 request_id=c741b661-12a9-4cfb-7591-f35ab2303676 source=http-debug vu=0
time="2021-08-10T21:22:14+02:00" level=warning msg="Request Failed" error="Get \"https://asdajkdahdqiuwhejkasdnakjdnadasdlkas.com\": lookup asdajkdahdqiuwhejkasdnakjdnadasdlkas.com: no such host" 
time="2021-08-10T21:22:14+02:00" level=warning msg="Request Failed" error="Get \"http://sdafsgdhfjg.com/\": lookup sdafsgdhfjg.com: no such host"
...
time="2021-08-10T21:22:16+02:00" level=warning msg="Request Failed" error="Get \"http://127.0.0.1:1/pesho\": dial: i/o timeout"
--- FAIL: TestErrorCodes (2.09s)
    --- FAIL: TestErrorCodes/Connection_refused_redirect (2.01s)
        request_test.go:2078:
                Error Trace:    request_test.go:2078
                Error:          Received unexpected error:
                                Error: wrong error: 'dial: i/o timeout' at <eval>:6:73(50)
                Test:           TestErrorCodes/Connection_refused_redirect
...
time="2021-08-10T21:22:20+02:00" level=warning msg="A batch request failed" error="invalid URL: parse \"https:// invalidurl.com\": invalid character \" \" in host name"
time="2021-08-10T21:22:20+02:00" level=warning msg="Request Failed" error="invalid URL: parse \"https:// test.k6.io\": invalid character \" \" in host name"
time="2021-08-10T21:22:20+02:00" level=warning msg="Request Failed" error="invalid URL: parse \"https:// test.k6.io\": invalid character \" \" in host name"
FAIL
FAIL    go.k6.io/k6/js/modules/k6/http  8.811s

and a lot of these

--- FAIL: TestConstantArrivalRateRunCorrectTiming (0.00s)
    --- FAIL: TestConstantArrivalRateRunCorrectTiming/segment_2/3:1_sequence_<nil> (2.02s)
        constant_arrival_rate_test.go:215:
                Error Trace:    constant_arrival_rate_test.go:215
                                                        common_test.go:40
                                                        minirunner.go:230
                                                        helpers.go:88
                                                        ramping_arrival_rate.go:549
                                                        asm_amd64.s:1371
                Error:          Max difference between 2021-08-10 21:22:14.4502656 +0200 CEST m=+0.307944801 and 2021-08-10 21:22:14.4654449 +0200 CEST m=+0.323124101 allowed is 12ms, but difference was -15.1793ms
                Test:           TestConstantArrivalRateRunCorrectTiming/segment_2/3:1_sequence_<nil>
                Messages:       6 expectedTime 300ms

What could be my issues? Is this normal for running the full test suite from the root dir?

Sorry if there’s details missing. And sorry for the long post. As I wrote, this is my first go project, so be gentle :sweat_smile:.

But big thanks in advance for helping out :slight_smile:!

2

Hi @SaberStrat , welcome to the community forum :tada:

The first error is known issue that we don’t know how to fix, but as we have nobody using macOS on the team and AFAIK this is only a failing test, the functionality still “mostly” works, we have not tried to figure it out. But you are welcome to do so.

The seconds one is because apparently windows error messages are localized (probably on other systems as well) which breaks the test. I don’t really have a good way to make this not the case I guess we can try to force using english error messages in tests :man_shrugging: .

The third one seems like possibly something with a firewall or something like that as we try to connect to 127.0.0.1:1 which should not be possible but I guess something (likely a firewall) might decide to just drop the whole connection at which point we will timeout.

The forth is a flaky test that … technically is a lot less flaky now than it was a year ago, but still fails every now and then.

You can just ignore them as long as your fix has nothing to do with them, we run the tests in the github actons where they in general don’t fail (except for the flaky one every now and then)

On point 1804: what is the fix that you are going to try to implement?

3

Thank you very much for the explanations!

I was wanting to play around with the InitContext’s compileImport() method’s newlines to see if it fixes the line number offset. But as your quick analysis showed, it’s probably a bit more involved after all. So I fear this one is too much of a steep learning curve for me to fix in reasonable time.

I’d love to help get it fixed sooner for my own load testing productivity’s sake. But given my (lack of) go experience, I think I’ll have to cope with working with one huge main.js script file for the time being :wink:, and just not let any other devs see my scripts :joy:.

1 Like