Connecting Alloy, loki and minio (K8s)

Grafana Alloy
,
1

I am having problem with Alloy, Loki, Prometheus and MiniO.

I can see the logs and data from Prometheus and Loki, but I’m not able to get any data to show up in MinIO. The connection between Loki and MinIO doesn’t seem to be working as expected, even though everything else is functioning correctly. I am using a external MiniO server, does anyone have a good values.yaml for this type of installation? Where I cant get Alloy(k8s-monitoring helm) + loki (grafana/loki helm) + Prometheus (kube-prometheus-stack helm) to connect together and send logs to minio.

2
  1. Please share your Loki configuration.
  2. What errors are you seeing when you attempted to use MinIO as storage for Loki?
3 
loki:
  auth_enabled: false
  commonConfig:
    replication_factor: 1
  schemaConfig:
    configs:
      - from: 2024-04-01
        store: tsdb
        object_store: s3
        schema: v13
        index:
          prefix: loki_index_
          period: 24h
  ingester:
    chunk_encoding: snappy
  tracing:
    enabled: true
  pattern_ingester:
      enabled: true
  limits_config:
    allow_structured_metadata: true
    volume_enabled: true
  ruler:
    enable_api: true
  querier:
    max_concurrent: 4

minio:
  enabled: true  

storage_config:
  aws:
    region: ---
    bucketnames:
      chunks: loki-chunks  
      ruler: loki-ruler 
      admin: loki-admin    
    s3forcepathstyle: true  
  s3:
    endpoint: http://-----:9000
    accessKeyId: -----  
    secretAccessKey: -----  
    s3ForcePathStyle: true  
    insecure: true  
    signatureVersion: v4  # MinIO använder v4 för signering

deploymentMode: SingleBinary
singleBinary:
  replicas: 1
  resources:
    limits:
      cpu: 0.5
      memory: 1Gi
    requests:
      cpu: 0.5
      memory: 1Gi
  extraEnv:
    - name: GOMEMLIMIT
      value: 750MiB
  tolerations:
    - key: "node-role.kubernetes.io/control-plane"
      operator: "Exists"
      effect: "NoSchedule"

chunksCache:
  writebackSizeLimit: 10MB
  enabled: false

resultsCache:
  writebackSizeLimit: 10MB
  enabled: false

test:
  enabled: false
lokiCanary:
  enabled: false

# Zero out replica counts of other deployment modes
backend:
  replicas: 0
read:
  replicas: 0
write:
  replicas: 0
ingester:
  replicas: 0
querier:
  replicas: 0
queryFrontend:
  replicas: 0
queryScheduler:
  replicas: 0
distributor:
  replicas: 0
compactor:
  replicas: 0
indexGateway:
  replicas: 0
bloomCompactor:
  replicas: 0
bloomGateway:
  replicas: 0
  1. I am not getting any errors, the installment seems fine but no data gets sent to my minio server. I only got a “loki_cluster_seed.json” to my bucket “loki-chunks”.
4

I don’t see anything obviously wrong. Are logs instead stored on Loki’s filesystem?

I’d also try to enable debug log and see if it shows anything, and also check your MinIO logs and see if there is any error there.

5

The rest of the installation I am running:

helm install prom prometheus-community/kube-prometheus-stack -n prod --set nodeExporter.enabled=false

helm install alloy grafana/k8s-monitoring -n prod --values values-alloy.yaml

helm install loki grafana/loki -n prod --values values-loki.yaml

cluster:
name: test-cluster

destinations:

clusterMetrics:
enabled: true

nodeMetrics:
enabled: true

clusterEvents:
enabled: true

logs:
enabled: true

kubeStateMetrics:
enabled: true

alloy-metrics:
enabled: true

alloy-singleton:
enabled: true

I am running longhorn, I think they are being stored locally there.