Extremely high memory on EC2 instance

Grafana k6
1

This is my first time using K6 and I’m loving it so far.

I need to run a load test with 36000 VUs. I keep running out of memory on an amazon EC2 instance during initialization. I’m generating memory reports using /usr/bin/time.

What could be the cause of this gigantic memory difference?
User scripts generated using postman-to-k6.

Mac OS side
Apple M1 Pro Sequoia 15.3.1 16gb
k6 v0.57.0 (go1.24.0, darwin/arm64)

It gets killed at 6061 VUs due to out of memory, as expected. /usr/bin/time -l k6 run script.js reports 3071 MB under maximum resident set size.

Full time log;

3221028864  maximum resident set size
                   0  average shared memory size
                   0  average unshared data size
                   0  average unshared stack size
            14416940  page reclaims
                1478  page faults
                   0  swaps
                   0  block input operations
                   0  block output operations
                  47  messages sent
                  61  messages received
               69648  signals received
              887282  voluntary context switches
             4450816  involuntary context switches
       2771119850002  instructions retired
       1076649572957  cycles elapsed
         71403542912  peak memory footprint

EC2 Instance
R6G.2XLarge 64GB running Amazon Linux 2023 LTS
k6 v0.57.0 (commit/50afd82c18, go1.23.6, linux/arm64)

(manually killed or the EC2 will hang) at 1077 VUs. /usr/bin/time -v k6 run script.js reports 10.57GB under Maximum resident set size.

Full time log;

User time (seconds): 73.19
        System time (seconds): 4.97
        Percent of CPU this job got: 733%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:10.65
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 11088080
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 3001623
        Voluntary context switches: 15405
        Involuntary context switches: 1817
        Swaps: 0
        File system inputs: 0
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 105
3

Hi @marvinmons55,

Welcome to our community forums! :party_popper:
I’m glad to read that you’re loving your first experience with k6 :grinning_face:

Generally speaking, I’d say that high memory consumption in k6 is usually because of one of the following two main reasons:

  • Large amount of metric samples, usually caused either by long (large duration) tests or extremely short iterations (like an empty default function, or without asynchronous operations at all).
  • Test data (like a JSON or CSV file used by the script) allocated in memory thousands of times, which usually happens when combining a large test (big number of VUs) and loading a data file in a not efficient way (e.g. not using SharedArray).

So, if you say this apparently happens during initialization, and considering it seems to be causing such high memory consumption only after ~a minute of execution, plus you’re indeed trying to run a considerably large test (big number of VUs), I’d say it’s likely the latter.

However, without actually seeing the test script code (or more or less how it looks), it’s difficult to completely confirm whether that’s the cause or something else. Therefore, I’d like to ask you: could you please share the relevant parts of your test script to understand better how it behaves?

Feel free to anonymize sensitive information like endpoints, IPs, data -like users, emails, etc-, of course. Just understanding more or less what it does (e.g. does it open a file?) , and how it does those operations (does it use the new open function? does it use SharedArray?) would help a lot.

I’ll be happy to bring you more assistance in such case.

Cheers! :person_bowing: