Running Loki in k8s, using loki-distributed helm chart.
Index is TSDB and using latest Loki version.
Was able to scale loki-distributed-querier by adding more machines and BytesProcessedPerSecond upto 24GB (15 machines[16 core, 32Gb machines]).
But once we add more machines 20, or 25, won’t see much difference in performace?, is BytesProcessedPerSecond depends upon any specific setting apart from max_concurrent.?.
thanks in advance.
There are several configurations that are related to parallelism. The one that’s directly related to querier is split_queries_by_interval. It is possible that at some point you can have more queriers than split queries to run (from a testing perspective). There is also a storage configuration that sets a limit to maximum number of chunks that can be read at the same time.
24GB per second processed is pretty impressive.
Followed recomended settings mentioned in the webinar, Best practices for configuring Grafana Loki
max_query_parallelism: 128
split_queries_by_interval: 15m
querier:
max_concurrent: 64
This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.