Long term log storage

[kylesplant]

Hi Loki Team/Community!

Hopefully this is an easy one to answer. We have Loki running in microservice mode, index is in DynamoDB and chunks in S3. Currently, we are retaining 4 weeks worth of logs and I have set lifecycle policies on the S3 bucket to automatically clean up stale logs.

We are at the point now where we would like to keep more logs in Loki. I assume a lot of users will want to keep logs for long periods of time for auditing and compliance reasons, and I am looking for the best way to handle this.

1. Are there any performance considerations if we are keeping more logs?

1 month vs 1 year vs 10 years?

2. If there are performance trade-offs, what kind of strategy can you recommend for backing up/restoring logs?

Let’s say I turn off lifecycle policy on the bucket, can Loki reindex files that already exist? Could I do this on-demand for a date range that I need to audit?

Would love to know how others have solved this.

Cheers
Kyle

[dror1212]

Can I ask what s3 service are you using? self managed one like ceph or something like amazon service? and if you try looking at graphs made of logs (if you did that), how are your performances for more than 30 days back?