Hello Loki team.
Why it is a bad idea to use Loki Compactor as a stateless solution? I know about marker file of Compactor, but I am not sure what we would lose if a file was dropped. Could you please explain the implications of using Loki Compactor as Stateful Set service?
Thank you for your assistance.
Marker file records the chunk files to be removed, if it’s gone then you will have some rogue chunk files that are supposedly to be removed but aren’t. It’s not critical to the operation of your Loki cluster. We don’t use stateful set for our compactor either (because we actually operate Loki in AWS ECS), and with the additional S3 lifecycle policy it’s not been a concern for us.