Hello all,
Apologies if this was asked and resolved before, I couldn’t find a solution for the same problem.
I have a question about using SharedArray on a scenario with a (relatively) large dataset. As suggested in the documentation, I am opening all the files I need during the init code of the scenario. And this is how I open the files in a loop that runs as long as the file count:
file: open(`../image/documents/${fileName}`, "b"),
This of course opens the file as a binary, and wrapping the original array that includes all the opened files with a SharedArray structure returns the object of type:
map[string]interface {} (I suppose this is a Go type?)
with the following error:
ERRO[0013] invalid type map[string]interface {}, expected string, []byte or ArrayBuffer
I have seen some discussions around the same problem, and tried some of the suggested workarounds like converting the object to an ArrayBuffer type, but that ends up with data inconsistency.
Another problem is that the init code calls the method that opens the files on each VU iteration, rather than once. Which, as expected, creates a huge memory usage and makes the process exhaust all the memory limits, hence failing the run.
So my questions are:
- Is there currently a way to open unstructured datasets using
SharedArray, hence allowing k6 to use the same files, and handle garbage collection properly? - If not, is there another way to open all the files just once, outside of the VU code, but allowing each VU to access the same exact array of opened files apart from
SharedArrays?
Thanks in advance for any help!