How to set AWS S3 bucket as object storage for Grafana Loki?

Grafana Loki
,
1

I am trying to get the storage for Grafana Loki set up in S3 bucket instead of local filesystem. Not seem to be able to make it work. I dont see any objects stored in the S3 bucket. Any help appreciated. I am using AWS as root IAM user.
Here is my loki-config.yaml:

auth_enabled: false

server:
  http_listen_port: 3100
  grpc_listen_port: 9096

common:
  instance_addr: 127.0.0.1
  path_prefix: loki
  storage:
    s3:
      bucketnames: my-bucket-name
      region: eu-central-1
  replication_factor: 1
  ring:
    kvstore:
      store: inmemory

query_range:
  results_cache:
    cache:
      embedded_cache:
        enabled: true
        max_size_mb: 100

schema_config:
  configs:
    - from: 2020-10-24
      store: boltdb-shipper
      object_store: filesystem
      schema: v11
      index:
        prefix: index_
        period: 24h

storage_config:
  boltdb_shipper:
    active_index_directory: /loki/boltdb-shipper-active
    cache_location: /loki/boltdb-shipper-cache
    cache_ttl: 24h         # Can be increased for faster performance over longer query periods, uses more disk space
    shared_store: s3
  aws:
    s3: s3://ABCDEFGHIJKLMOP:0909yyhskdhfsdfha82745475skfhsdkf@eu-central-1/my-bucket-name

compactor:
  working_directory: /loki/boltdb-shipper-compactor
  shared_store: aws

chunk_store_config:
  max_look_back_period: 48h
limits_config:
  retention_period: 48h

table_manager:
  retention_deletes_enabled: true
  retention_period: 48h

ruler:
  alertmanager_url: http://localhost:9093

# By default, Loki will send anonymous, but uniquely-identifiable usage and configuration
# analytics to Grafana Labs. These statistics are sent to https://stats.grafana.org/
#
# Statistics help us better understand how Loki is used, and they show us performance
# levels for most users. This helps us prioritize features and documentation.
# For more information on what's sent, look at
# https://github.com/grafana/loki/blob/main/pkg/usagestats/stats.go
# Refer to the buildReport method to see what goes into a report.
#
# If you would like to disable reporting, uncomment the following lines:
analytics:
 reporting_enabled: false

and here is the IAM policy I am attaching with the S3 bucket:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "LokiStorage",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789098:root"
            },
            "Action": [
                "s3:ListBucket",
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::my-bucket-name",
                "arn:aws:s3:::my-bucket-name/*"
            ]
        }
]
}

And I launch it via docker container like so:

docker run -d --name loki --restart unless-stopped -v $(pwd):/mnt/config -p 3100:3100 grafana/loki:2.9.1 -config.file=/mnt/config/loki-config.yaml

The container runs fine, but no objects get stored in S3 bucket. Any help in getting the S3 object storage working for Grafana Loki will be appreciated.

1 Like