Hi,
We are testing Loki to feed in logs that are mostly legacy/unstructured. I guess only the hostname can really be indexed. It is around 10-25MB uncompressed per day but we intend to cover many years.
Is this in general a bad idea? We do see Loki consuming a lot of memory after various searches and this with only a fraction of the data. Around 1 month.
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1340 root 20 0 36.2g 35.3g 78404 S 0.0 56.4 39:21.14 loki
Regards, Bjorn