K6 runners fail on a large script

Grafana k6 OSS Support
1

Hi all,

I have written a script for load testing my service The size of the script is > 256kB. I am facing a peculiar problem where in I if I supply parallelism as 5 to k6 operator out of 5 pods 2 pods fail with error :

time="2024-12-13T07:25:16Z" level=debug msg="Logger format: TEXT"
time="2024-12-13T07:25:16Z" level=debug msg="k6 version: v0.50.0 (go1.21.10, linux/amd64)"
time="2024-12-13T07:25:16Z" level=debug msg="Resolving and reading test '/test/authentication-utils.js'..."
time="2024-12-13T07:25:16Z" level=debug msg=Loading... moduleSpecifier="file:///test/authentication-utils.js" originalModuleSpecifier=/test/authentication-utils.js
time="2024-12-13T07:25:16Z" level=debug msg="Everything has finished, exiting k6 with an error!" error="The moduleSpecifier \"/test/authentication-utils.js\" couldn't be found on local disk. Make sure that you've specified the right path to the file. If you're running k6 using the Docker image make sure you have mounted the local directory (-v /local/path/:/inside/docker/path) containing your script and modules so that they're accessible by k6 from inside of the container, see https://grafana.com/docs/k6/latest/using-k6/modules/#using-local-modules-with-docker."
time="2024-12-13T07:25:16Z" level=error msg="The moduleSpecifier \"/test/authentication-utils.js\" couldn't be found on local disk. Make sure that you've specified the right path to the file. If you're running k6 using the Docker image make sure you have mounted the local directory (-v /local/path/:/inside/docker/path) containing your script and modules so that they're accessible by k6 from inside of the container, see https://grafana.com/docs/k6/latest/using-k6/modules/#using-local-modules-with-docker."

but the other 3 pods have the following log :

time="2024-12-13T07:25:16Z" level=debug msg="Logger format: TEXT"
time="2024-12-13T07:25:16Z" level=debug msg="k6 version: v0.50.0 (go1.21.10, linux/amd64)"
time="2024-12-13T07:25:16Z" level=debug msg="Resolving and reading test '/test/authentication-utils.js'..."
time="2024-12-13T07:25:16Z" level=debug msg=Loading... moduleSpecifier="file:///test/authentication-utils.js" originalModuleSpecifier=/test/authentication-utils.js
time="2024-12-13T07:25:16Z" level=debug msg="'/test/authentication-utils.js' resolved to 'file:///test/authentication-utils.js' and successfully loaded 993727 bytes!"
time="2024-12-13T07:25:16Z" level=debug msg="Gathering k6 runtime options..."
time="2024-12-13T07:25:16Z" level=debug msg="Initializing k6 runner for '/test/authentication-utils.js' (file:///test/authentication-utils.js)..."
time="2024-12-13T07:25:16Z" level=debug msg="Detecting test type for..." test_path="file:///test/authentication-utils.js"
time="2024-12-13T07:25:16Z" level=debug msg="Trying to load as a JS test..." test_path="file:///test/authentication-utils.js"
time="2024-12-13T07:25:16Z" level=warning msg="The source for `file:///test/authentication-utils.js` needs to go through babel but is over 256000 bytes. For performance reasons source map support will be disabled for this particular file."
time="2024-12-13T07:25:58Z" level=debug msg="Babel: Transformed" t=41.088758056s
time="2024-12-13T07:25:58Z" level=debug msg="Runner successfully initialized!"
time="2024-12-13T07:25:58Z" level=debug msg="Parsing CLI flags..."
time="2024-12-13T07:25:58Z" level=debug msg="Consolidating config layers..."
time="2024-12-13T07:25:58Z" level=debug msg="Parsing thresholds and validating config..."
time="2024-12-13T07:25:58Z" level=debug msg="Initializing the execution scheduler..."
time="2024-12-13T07:25:58Z" level=debug msg="Starting 1 outputs..." component=output-manager
time="2024-12-13T07:25:58Z" level=debug msg=Starting... component=metrics-engine-ingester
time="2024-12-13T07:25:58Z" level=debug msg="Started!" component=metrics-engine-ingester
time="2024-12-13T07:25:58Z" level=debug msg="     execution: local\n        script: /test/authentication-utils.js\n        output: -\n\n     scenarios: (20.00%) 1 scenario, 3200 max VUs, 9m30s max duration (incl. graceful stop):\n              * ramping_arrival_rate: Up to 1000.00 iterations/s for 9m0s over 9 stages (maxVUs: 200-3200, gracefulStop: 30s)\n\n"
time="2024-12-13T07:25:58Z" level=debug msg="Trapping interrupt signals so k6 can handle them gracefully..."
time="2024-12-13T07:25:58Z" level=debug msg="Starting the REST API server on 0.0.0.0:6565"
time="2024-12-13T07:25:58Z" level=debug msg="Starting emission of VUs and VUsMax metrics..."
time="2024-12-13T07:25:58Z" level=debug msg="Start of initialization" executorsCount=1 neededVUs=200 phase=execution-scheduler-init

as not all the pods start my test does not run

following is the configuration of runners :

apiVersion: k6.io/v1alpha1
          kind: TestRun
          metadata:
            name: load-test
          spec:
            parallelism: {{workflow.parameters.parallelism}}
            script:
              volumeClaim:
                name: argo-pvc
                file: {{workflow.parameters.scriptFileName}}
            arguments: --tag testid={{workflow.parameters.testId}} --log-output=stdout --verbose
            runner:
              livenessProbe:
                httpGet:
                  path: /v1/status
                  port: 6565
                initialDelaySeconds: 60
                periodSeconds: 30
                timeoutSeconds: 25
                failureThreshold: 6
                successThreshold: 1
              readinessProbe:
                httpGet:
                  path: /v1/status
                  port: 6565
                initialDelaySeconds: 60
                periodSeconds: 30
                timeoutSeconds: 25
                failureThreshold: 6
                successThreshold: 1  
              securityContext:
                runAsNonRoot: false
                runAsUser: 0  # Run as root
                fsGroup: 0    # Set file system group to root
              resources:
                limits:
                  cpu: 1000m
                  memory: 2048Mi
                requests:
                  cpu: 1000m
                  memory: 2048Mi  
              volumes:
                - hostPath:
                    path: /var/log/kaarya
                  name: logfolder
              volumeMounts:
                - mountPath: /tmp
                  name: logfolder
              image: 578061096415.dkr.ecr.us-east-1.amazonaws.com/load-test:1.0.0
              imagePullSecrets:
                - name: dockercred
              imagePullPolicy: Always

can some pls help me as to what is going on here

Screenshot 2024-12-13 at 1.05.58 PM
Screenshot 2024-12-13 at 1.05.58 PM2956×1102 286 KB

2

Hi @vipulkhullar

I believe the main issue describes by this error:
The moduleSpecifier \"/test/authentication-utils.js\" couldn't be found on local disk. Make sure that you've specified the right path to the file. If you're running k6 using the Docker image make sure you have mounted the local directory (-v /local/path/:/inside/docker/path) containing your script and modules so that they're accessible by k6 from inside of the container, see https://grafana.com/docs/k6/latest/using-k6/modules/#using-local-modules-with-docker.

So test run failed because the file /test/authentication-utils.js wasn’t found. So I’d recommend checking if it’s actually packing into the k6’s archive.

As a sidenote, which note related to an actual issue, I also see that you use the k6 v0.50 which is old, and I recommend upgrading to the latest available, like k6 v0.55 the version where we did a couple of improvements that could be notable for the running of large scripts.

Hope that answers!

3

Hi @olegbespalov , I will update the grafana k6 version but why does the error come only on 2 pods out of 5 as I have mentioned above ?
for the other 3 pods file was found from the same volume claim what could be the reason ?

time="2024-12-13T07:25:16Z" level=debug msg="Logger format: TEXT"
time="2024-12-13T07:25:16Z" level=debug msg="k6 version: v0.50.0 (go1.21.10, linux/amd64)"
time="2024-12-13T07:25:16Z" level=debug msg="Resolving and reading test '/test/authentication-utils.js'..."
time="2024-12-13T07:25:16Z" level=debug msg=Loading... moduleSpecifier="file:///test/authentication-utils.js" originalModuleSpecifier=/test/authentication-utils.js
time="2024-12-13T07:25:16Z" level=debug msg="'/test/authentication-utils.js' resolved to 'file:///test/authentication-utils.js' and successfully loaded 993727 bytes!"
time="2024-12-13T07:25:16Z" level=debug msg="Gathering k6 runtime options..."
time="2024-12-13T07:25:16Z" level=debug msg="Initializing k6 runner for '/test/authentication-utils.js' (file:///test/authentication-utils.js)..."
time="2024-12-13T07:25:16Z" level=debug msg="Detecting test type for..." test_path="file:///test/authentication-utils.js"
time="2024-12-13T07:25:16Z" level=debug msg="Trying to load as a JS test..." test_path="file:///test/authentication-utils.js"
time="2024-12-13T07:25:16Z" level=warning msg="The source for `file:///test/authentication-utils.js` needs to go through babel but is over 256000 bytes. For performance reasons source map support will be disabled for this particular file."
time="2024-12-13T07:25:58Z" level=debug msg="Babel: Transformed" t=41.088758056s
time="2024-12-13T07:25:58Z" level=debug msg="Runner successfully initialized!"
time="2024-12-13T07:25:58Z" level=debug msg="Parsing CLI flags..."
time="2024-12-13T07:25:58Z" level=debug msg="Consolidating config layers..."
time="2024-12-13T07:25:58Z" level=debug msg="Parsing thresholds and validating config..."
time="2024-12-13T07:25:58Z" level=debug msg="Initializing the execution scheduler..."
time="2024-12-13T07:25:58Z" level=debug msg="Starting 1 outputs..." component=output-manager
time="2024-12-13T07:25:58Z" level=debug msg=Starting... component=metrics-engine-ingester
time="2024-12-13T07:25:58Z" level=debug msg="Started!" component=metrics-engine-ingester
time="2024-12-13T07:25:58Z" level=debug msg="     execution: local\n        script: /test/authentication-utils.js\n        output: -\n\n     scenarios: (20.00%) 1 scenario, 3200 max VUs, 9m30s max duration (incl. graceful stop):\n              * ramping_arrival_rate: Up to 1000.00 iterations/s for 9m0s over 9 stages (maxVUs: 200-3200, gracefulStop: 30s)\n\n"
time="2024-12-13T07:25:58Z" level=debug msg="Trapping interrupt signals so k6 can handle them gracefully..."
time="2024-12-13T07:25:58Z" level=debug msg="Starting the REST API server on 0.0.0.0:6565"
time="2024-12-13T07:25:58Z" level=debug msg="Starting emission of VUs and VUsMax metrics..."
time="2024-12-13T07:25:58Z" level=debug msg="Start of initialization" executorsCount=1 neededVUs=200 phase=execution-scheduler-init
4

@vipulkhullar can you please validate (outside the k6) that all volumes mounted and files are discoverable, I mean like list all /test catalog?

5

its showing the following :
i have created a dummy container and mounted /test directory of it to the the pvc

and i am pushing my script in the pv via following

 - name: fetch-script
      inputs: {}
      outputs: {}
      metadata: {}
      container:
        name: ''
        image: alpine:latest
        command:
          - sh
          - '-c'
        args:
          - >
            mkdir -p /test && apk add --no-cache curl && curl -X GET
            "{{workflow.parameters.gitlink}}" >
            /test/{{workflow.parameters.scriptFileName}}
        resources: {}
        volumeMounts:
          - name: stress-test-volume
            mountPath: /test
      volumes:
        - name: stress-test-volume
          persistentVolumeClaim:
            claimName: argo-pvc

Screenshot 2024-12-13 at 10.31.07 PM
Screenshot 2024-12-13 at 10.31.07 PM1308×670 27.7 KB

6

any updates ?