Loki/promtail (S3+Dynamodb) Disaster recovery

Hello community,

I’m bother you all with this topic, in order to see if someone of you had the same issue here related to disaster recovery. I need to have a reliable way to recover logs in case of a problem (system corruption / update fail & fail to rollback / etc)…

I’m using S3 for chunks and dynamo for indexes loki+promtail are deployed in Kubernetes (everything working fine so far).

My S3 looks like:

In the root:
. file_seed.json
. fake directory

The fake directory is filled with files like 1030fbb16a164dc9_182c0c9d66d_182c0cb133d_a74882b6 with type Targa image data .

The question is should i backup all this S3 bucket ? how can i restore this in any other place in case of disaster?

Thanks in advance for all the help.
Have a great day! !