Loki Helm Simple Scalable Deployment CrashLoopBackOff

Trying to setup Simple Scalable loki using instructions provided at Install the simple scalable Helm chart | Grafana Loki documentation but few containers are in CrashLoopBackOff. Using minikube to setup 4 nodes k8s cluster using docker driver. Post deploy, loki-backend continues to fail, error below:

{"time": "2025-03-14T17:14:55.166773+00:00", "level": "INFO", "msg": "Starting collector"}
{"time": "2025-03-14T17:14:55.167211+00:00", "level": "WARNING", "msg": "No folder annotation was provided, defaulting to k8s-sidecar-target-directory"}
{"time": "2025-03-14T17:14:55.167536+00:00", "level": "INFO", "msg": "Loading incluster config ..."}
{"time": "2025-03-14T17:14:55.168626+00:00", "level": "INFO", "msg": "Config for cluster api at 'https://10.96.0.1:443' loaded..."}
{"time": "2025-03-14T17:14:55.168857+00:00", "level": "INFO", "msg": "Unique filenames will not be enforced."}
{"time": "2025-03-14T17:14:55.169045+00:00", "level": "INFO", "msg": "5xx response content will not be enabled."}

loki-write components continue to fail due to:

mkdir /var/loki/tsdb-shipper-active: permission denied
error initialising module: store
github.com/grafana/dskit/modules.(*Manager).initModule
	/src/loki/vendor/github.com/grafana/dskit/modules/modules.go:138
github.com/grafana/dskit/modules.(*Manager).InitModuleServices
	/src/loki/vendor/github.com/grafana/dskit/modules/modules.go:108
github.com/grafana/loki/v3/pkg/loki.(*Loki).Run
	/src/loki/pkg/loki/loki.go:495
main.main
	/src/loki/cmd/loki/main.go:129
runtime.main
	/usr/local/go/src/runtime/proc.go:272
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1700
level=info ts=2025-03-14T19:02:07.424455349Z caller=main.go:126 msg="Starting Loki" version="(version=3.4.2, branch=release-3.4.x, revision=4fa045d3)"
level=info ts=2025-03-14T19:02:07.424497684Z caller=main.go:127 msg="Loading configuration file" filename=/etc/loki/config/config.yaml
level=info ts=2025-03-14T19:02:07.429396054Z caller=server.go:351 msg="server listening on addresses" http=[::]:3100 grpc=[::]:9095
level=info ts=2025-03-14T19:02:07.445468405Z caller=memberlist_client.go:439 msg="Using memberlist cluster label and node name" cluster_label= node=loki-write-2-80419bb3
level=info ts=2025-03-14T19:02:07.446476443Z caller=memberlist_client.go:549 msg="memberlist fast-join starting" nodes_found=1 to_join=4
level=error ts=2025-03-14T19:02:07.451518056Z caller=resolver.go:100 msg="failed to lookup SRV records" host=_memcached-client._tcp.loki-chunks-cache.default.svc err="lookup _memcached-client._tcp.loki-chunks-cache.default.svc on 10.96.0.10:53: no such host"
level=info ts=2025-03-14T19:02:07.45987262Z caller=shipper.go:165 index-store=tsdb-2024-04-01 msg="starting index shipper in WO mode"
level=error ts=2025-03-14T19:02:07.460783873Z caller=log.go:216 msg="error running loki" err="mkdir /var/loki/tsdb-shipper-active: permission denied\nerror initialising module: store\ngithub.com/grafana/dskit/modules.(*Manager).initModule\n\t/src/loki/vendor/github.com/grafana/dskit/modules/modules.go:138\ngithub.com/grafana/dskit/modules.(*Manager).InitModuleServices\n\t/src/loki/vendor/github.com/grafana/dskit/modules/modules.go:108\ngithub.com/grafana/loki/v3/pkg/loki.(*Loki).Run\n\t/src/loki/pkg/loki/loki.go:495\nmain.main\n\t/src/loki/cmd/loki/main.go:129\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:272\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1700"

Overall status:

NAME                            READY   STATUS             RESTARTS        AGE
loki-backend-0                  1/2     CrashLoopBackOff   26 (66s ago)    110m
loki-backend-1                  1/2     CrashLoopBackOff   26 (19s ago)    110m
loki-backend-2                  1/2     CrashLoopBackOff   26 (101s ago)   110m
loki-canary-5n5q7               1/1     Running            0               110m
loki-canary-8mrzw               1/1     Running            0               110m
loki-canary-f5cgr               1/1     Running            0               110m
loki-canary-lrzpg               1/1     Running            0               110m
loki-chunks-cache-0             0/2     Pending            0               110m
loki-gateway-5858bd5949-vz47x   1/1     Running            0               110m
loki-minio-0                    0/1     CrashLoopBackOff   26 (69s ago)    110m
loki-read-5df6cf8c68-9h27r      0/1     Running            0               110m
loki-read-5df6cf8c68-mktw6      0/1     Running            0               110m
loki-read-5df6cf8c68-szsmq      0/1     Running            0               110m
loki-results-cache-0            2/2     Running            0               110m
loki-write-0                    0/1     CrashLoopBackOff   26 (102s ago)   110m
loki-write-1                    0/1     CrashLoopBackOff   26 (12s ago)    110m
loki-write-2                    0/1     CrashLoopBackOff   26 (79s ago)    110m

Any recommendations how to get the deployment to work? Thanks.

Well, I am assuming you are using minio as backend, since your minio pod isn’t running naturally your loki pods won’t be able to start because there is no backend available.

Thanks for your response, I moved from docker to podman. I see minio as running now, but backend and write are still in CrashLoopBackOff.

NAME                            READY   STATUS             RESTARTS         AGE
loki-backend-0                  1/2     CrashLoopBackOff   21 (4m37s ago)   87m
loki-backend-1                  1/2     CrashLoopBackOff   21 (4m40s ago)   87m
loki-backend-2                  1/2     CrashLoopBackOff   21 (4m4s ago)    87m
loki-canary-8qbs2               1/1     Running            0                87m
loki-canary-gj9vb               1/1     Running            1 (87m ago)      87m
loki-canary-n4w5m               1/1     Running            0                87m
loki-canary-nqlfw               1/1     Running            0                87m
loki-chunks-cache-0             0/2     Pending            0                87m
loki-gateway-5858bd5949-c8d52   0/1     CrashLoopBackOff   22 (3m11s ago)   87m
loki-minio-0                    1/1     Running            0                87m
loki-read-5df6cf8c68-2q9ds      0/1     Running            0                87m
loki-read-5df6cf8c68-57qq6      0/1     Running            1 (87m ago)      87m
loki-read-5df6cf8c68-twwtz      0/1     Running            0                87m
loki-results-cache-0            2/2     Running            0                87m
loki-write-0                    0/1     CrashLoopBackOff   21 (86s ago)     87m
loki-write-1                    0/1     CrashLoopBackOff   21 (75s ago)     87m
loki-write-2                    0/1     CrashLoopBackOff   21 (67s ago)     87m

What are the errors you see now?

loki-write reports permission denied

level=info ts=2025-03-15T00:42:04.739646728Z caller=main.go:126 msg="Starting Loki" version="(version=3.4.2, branch=release-3.4.x, revision=4fa045d3)"
level=info ts=2025-03-15T00:42:04.739729847Z caller=main.go:127 msg="Loading configuration file" filename=/etc/loki/config/config.yaml
level=info ts=2025-03-15T00:42:04.742366979Z caller=server.go:351 msg="server listening on addresses" http=[::]:3100 grpc=[::]:9095
level=info ts=2025-03-15T00:42:04.744158421Z caller=memberlist_client.go:439 msg="Using memberlist cluster label and node name" cluster_label= node=loki-write-2-27e869fd
level=info ts=2025-03-15T00:42:04.744495331Z caller=memberlist_client.go:549 msg="memberlist fast-join starting" nodes_found=1 to_join=4
mkdir /var/loki/tsdb-shipper-active: permission denied
error initialising module: store
github.com/grafana/dskit/modules.(*Manager).initModule
	/src/loki/vendor/github.com/grafana/dskit/modules/modules.go:138
github.com/grafana/dskit/modules.(*Manager).InitModuleServices
	/src/loki/vendor/github.com/grafana/dskit/modules/modules.go:108
github.com/grafana/loki/v3/pkg/loki.(*Loki).Run
	/src/loki/pkg/loki/loki.go:495
main.main
	/src/loki/cmd/loki/main.go:129
runtime.main
	/usr/local/go/src/runtime/proc.go:272
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1700
level=error ts=2025-03-15T00:42:14.746832047Z caller=memcached_client.go:188 msg="error setting memcache servers to host" host= err="lookup SRV records \"_memcached-client._tcp.loki-chunks-cache.default.svc\": lookup _memcached-client._tcp.loki-chunks-cache.default.svc on 10.96.0.10:53: dial udp 10.96.0.10:53: i/o timeout"
level=info ts=2025-03-15T00:42:14.760019589Z caller=shipper.go:165 index-store=tsdb-2024-04-01 msg="starting index shipper in WO mode"
level=error ts=2025-03-15T00:42:14.760552014Z caller=log.go:216 msg="error running loki" err="mkdir /var/loki/tsdb-shipper-active: permission denied\nerror initialising module: store\ngithub.com/grafana/dskit/modules.(*Manager).initModule\n\t/src/loki/vendor/github.com/grafana/dskit/modules/modules.go:138\ngithub.com/grafana/dskit/modules.(*Manager).InitModuleServices\n\t/src/loki/vendor/github.com/grafana/dskit/modules/modules.go:108\ngithub.com/grafana/loki/v3/pkg/loki.(*Loki).Run\n\t/src/loki/pkg/loki/loki.go:495\nmain.main\n\t/src/loki/cmd/loki/main.go:129\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:272\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1700"

loki-backend reports

Defaulted container "loki-sc-rules" out of: loki-sc-rules, loki
{"time": "2025-03-14T19:14:48.718809+00:00", "level": "INFO", "msg": "Starting collector"}
{"time": "2025-03-14T19:14:48.719100+00:00", "level": "WARNING", "msg": "No folder annotation was provided, defaulting to k8s-sidecar-target-directory"}
{"time": "2025-03-14T19:14:48.719360+00:00", "level": "INFO", "msg": "Loading incluster config ..."}
{"time": "2025-03-14T19:14:48.720284+00:00", "level": "INFO", "msg": "Config for cluster api at 'https://10.96.0.1:443' loaded..."}
{"time": "2025-03-14T19:14:48.720439+00:00", "level": "INFO", "msg": "Unique filenames will not be enforced."}
{"time": "2025-03-14T19:14:48.720541+00:00", "level": "INFO", "msg": "5xx response content will not be enabled."}

@tonyswumac : Any thoughts on this? I am pretty sure I am following the documentation, but the cluster won’t turn healthy.

You have some permission issue in there. I would probably start from there.