what is the impact of increasing the chunk_target_size from 1.5mb to 3mb or even higher
Hi,
Did you have a look at this paragraph ?
Target Chunk Size
PR 1406 added the chunk_target_size configuration, which instructs Loki to try to fill chunks to the target size before flushing. This allows you to create bigger chunks than before, which can help reduce overhead on storage systems as well as overhead on querying. We are currently targeting 1572864bytes (1.5MB) chunks, which is helping our GCS burden. But it has also caused a number of problems for Memcached, which does not like storing objects this big. We will be looking soon to see if Redis performs better with objects of this size.
Hope it helps.
Good Luck
1 Like
@wlargou found a very helpful note indeed!
A few other considerations, chunks are downloaded for queries as well but are done in batches to help control memory usage.
I don’t see any problems really increase from 1.5MB to 3MB but the higher you go the more memory your queriers will consume fulfilling queries.
I would say feel free to experiment here if you are able to and if you learn anything interesting please respond back with what you find!
This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.