Issue Description:
We are currently facing a substantial issue with our Loki Ingester cluster, where there is a highly uneven distribution of logs across multiple instances. The default load balancing, which seems to be primarily based on the number of streams, is not effectively distributing the workload.
My question is twofold:
First, are there alternative load balancing strategies available apart from the one relying solely on the count of streams? We’re seeking a mechanism that could potentially take into account other factors, such as the volume of logs per stream or the overall resource consumption, to ensure a more efficient distribution.
Secondly, are there any configurations, optimizations, or best practices we can implement to enhance the load balancing and thereby promote a more equitable distribution of logging tasks among the ingesters? Our primary objective is to optimize resource utilization and maintain stability within our logging infrastructure.
Any guidance, suggestions, or pointers towards relevant documentation would be immensely appreciated. Thank you!