Issues with Loki 3.0.0


I am trying to install Loki 3.0.0 via helm chart simple-scalable. I have configured the storage via s3 bucket no mimir. The read pods are failing to start and I have the following in the logs:

level=info ts=2024-04-17T13:47:15.176886748Z caller=frontend.go:445 msg="not ready: number of schedulers this worker is connected to is 0"
level=error ts=2024-04-17T13:47:15.709284242Z caller=resolver.go:87 index-store=tsdb-2024-01-01 msg="failed to lookup IP addresses" host=loki-backend-headless.monitoring.svc.cluster.local err="lookup loki-backend-headless.monitoring.svc.cluster.local on no such host"
level=warn ts=2024-04-17T13:47:15.709323471Z caller=resolver.go:134 index-store=tsdb-2024-01-01 msg="IP address lookup yielded no results. No host found or no addresses found" host=loki-backend-headless.monitoring.svc.cluster.local
level=error ts=2024-04-17T13:47:15.716475483Z caller=ring_watcher.go:56 component=frontend-scheduler-worker msg="error getting addresses from ring" err="empty ring"
level=error ts=2024-04-17T13:47:17.717013145Z caller=ring_watcher.go:56 component=querier component=querier-scheduler-worker msg="error getting addresses from ring" err="empty ring"

any idea how to fix it?

Looks like your service discovery is not working. I’d try to fix this first:

level=error ts=2024-04-17T13:47:15.709284242Z caller=resolver.go:87 index-store=tsdb-2024-01-01 msg=“failed to lookup IP addresses” host=loki-backend-headless.monitoring.svc.cluster.local err=“lookup loki-backend-headless.monitoring.svc.cluster.local on no such host”