Loki Helm installation failure in eks Anywhere cluster

Grafana Loki
1

I am trying to install a Grafana Loki in an eks anywhere cluster with helm. The loki backend , read and write pods are not starting up . I cannot see to figure out what is missing here?
I have attached the logs and the values.yaml file as well.
Can anyone let me know what I am missing here?

Chart version: 5.44.4
Loki version: 2.9.4

Grafana Loki Status

NAME                                                   READY   STATUS        RESTARTS   AGE
loki-backend-0                                         0/2     Pending       0          34s
loki-gateway-77bdbc456b-rgd7z                          1/1     Running       0          34s
loki-read-7fd5877c55-2d7jl                             0/1     Running       0          34s
loki-write-0                                           0/1     Pending       0          34s

Logs out put

loki-backend-0

k logs loki-backend-0
Defaulted container "loki-sc-rules" out of: loki-sc-rules, loki

loki-gateway-77bdbc456b-rgd7z

k logs loki-gateway-77bdbc456b-rgd7z
192.168.1.123 - - [18/Mar/2024:22:07:03 +0000]  200 "GET / HTTP/1.1" 2 "-" "kube-probe/1.29" "-"

loki-read-7fd5877c55-2d7jl

k logs loki-read-7fd5877c55-2d7jl
level=info ts=2024-03-18T22:08:23.727217626Z caller=frontend.go:316 msg="not ready: number of schedulers this worker is connected to is 0"
level=warn ts=2024-03-18T22:08:24.142687459Z caller=dns_resolver.go:225 msg="failed DNS A record lookup" err="lookup query-scheduler-discovery.default.svc.cluster.local. on 10.96.0.10:53: no such host"
level=warn ts=2024-03-18T22:08:25.490268293Z caller=dns_resolver.go:225 msg="failed DNS A record lookup" err="lookup query-scheduler-discovery.default.svc.cluster.local. on 10.96.0.10:53: no such host"

loki-write-0

k loki-write-0

Loki values.yaml


# -- Overrides the chart's computed fullname
fullnameOverride: grafana-loki


loki:
 
  auth_enabled: false

  # -- Limits config
  limits_config:
    reject_old_samples: true
    reject_old_samples_max_age: 168h
    max_cache_freshness_per_query: 10m
    split_queries_by_interval: 15m

  # -- Storage config. Providing this will automatically populate all necessary storage configs in the templated config.
  storage:
    bucketNames:
      chunks: loki.test-bucket.com
      ruler: loki.test-bucket.com
      admin: loki.test-bucket.com
    type: s3
    s3:
      endpoint: https://s3-eu-west-1.amazonaws.com
      region: eu-west-1
     secretAccessKey: <MY_SECRET_KEY>
      accessKeyId: <MY_ACCESS_KEY>
      signatureVersion: null
      s3ForcePathStyle: false
      insecure: false
  schema_config:
      configs:
        - from: 2024-03-13
          object_store: s3
          index:
            prefix: index_
            period: 24h

serviceAccount:
  create: true
  name: grafana-loki-sa
  automountServiceAccountToken: true

# -- Section for configuring optional Helm test
test:
  enabled: false
monitoring:
  dashboards:
    enabled: false
  rules:
    enabled: false
  serviceMonitor:
    enabled: false
  selfMonitoring:
    enabled: false
    grafanaAgent:
      installOperator: false
  lokiCanary:
    enabled: false
   
    
# Configuration for the write pod(s)
write:
  replicas: 1
  autoscaling:
    enabled: false
    minReplicas: 1
    maxReplicas: 6
    targetCPUUtilizationPercentage: 60
    behavior:
      scaleUp:
        policies:
          - type: Pods
            value: 1
            periodSeconds: 900
      scaleDown:
        policies:
          - type: Pods
            value: 1
            periodSeconds: 1800
        stabilizationWindowSeconds: 3600

tableManager:
  enabled: false



# Configuration for the read pod(s)
read:
  replicas: 1
  autoscaling:
    enabled: false
    minReplicas: 1
    maxReplicas: 6
    targetCPUUtilizationPercentage: 60
   

# Configuration for the backend pod(s)
backend:
  replicas: 1
  autoscaling:
    enabled: false
    minReplicas: 1
    maxReplicas: 6
    targetCPUUtilizationPercentage: 60
   
# Configuration for the memberlist service
memberlist:
  service:
    publishNotReadyAddresses: true

# Configuration for the gateway
gateway:
  enabled: true
  replicas: 1
  verboseLogging: true
  autoscaling:
    enabled: false
    minReplicas: 1
    maxReplicas: 3
    targetCPUUtilizationPercentage: 60

  
minio:
  enabled: false
2

FYI - The Loki Helm charts was broken today and a couple of fixes have since been merged.