Hi there,
where can I read something about the throughput (MB/sec) a single loki instance can support? Is there any performance attribute do you suggest to monitor the actual loki performance to understand when is needed to scale out?
Thanks
Hi,
I suggest to try this dashboard :
Good Luck
I can only really give you some round numbers we haven’t done much benchmarking.
We generally keep the write volume at about 5MB/sec per distributor (as measured by loki_distributor_bytes_received_total) and scale horizontally to keep it around this amount, the ingesters see probably 10MB/sec because of replication done by the distributors.
We try to target about 10,000 to 15,000 active streams per ingester (measured by loki_ingester_memory_streams), with 15GB memory limits on our ingester pods, typically trying to keep the used memory about half of the limits.
Hope this helps some!
2 Likes
@ewelch The horizontal scaling is happening auto or is it done manually ? Just curious if we can add autoscaling in our current stack based on these numbers.
Is there any impact/cons of scaling down nodes ?