Is there a way to prevent the issue that occurs in Loki, particularly with the Loki distributor? The issue arises when the distributor is unable to push logs to the ingester due to unhealthy ingesters in the ring. These unhealthy ingesters are not visible using kubectl
commands unless you port-forward the distributor pod and check the /ring
endpoint. We are encountering this problem quite frequently now.
We are using helm chart version 6.12