External compactor

Hello!

Please, help to understand the deployment case.

I`m deploying Loki in monolithic mode in cluster, summary on 3 nodes.
Sometimes i got problem, that compactor is overloading Loki container because of inner processes.

Now i want to deploy Loki as example below:
loki-host11 - Loki container (without compactor) + Compactor (without other services Loki)
loki-host02 - Loki container (without compactor)
loki-host03 - Loki container (without compactor)

its possible? How to configure it? Any examples? Didn`t see any info about this in documentation.

P.S. Chunk-store - S3 (MinIO); Not possible to deploy Loki in k8s.

Perhaps consider going with simple scalable mode, see Loki deployment modes | Grafana Loki documentation.

Yeah, i understand that its better way to deploy on simple scalable mode, but as i write above - not possible to deploy in k8s
So, thats why i am asking about manual splitting services

Not sure where you got that from, it is absolutely possible to deploy Loki in simple scalable mode in K8S. See Loki deployment modes | Grafana Loki documentation

Hello, dear
I mean that i dont have k8s in prod, so its not possible for me to deploy it in k8s.

Thats why i am asking about splitting microservices manually like this
loki-host11 - Loki container (without compactor) + Compactor (without other services Loki)
loki-host02 - Loki container (without compactor)
loki-host03 - Loki container (without compactor)

What are you using as container platform?

Remember, K8S is, at its core, a container platform. many things that you can do on K8S can be done in ECS or even Docker Swarm. You don’t have read-to-use helm chart from community, sure, but it’s possible. We run our Loki cluster with AWS ECS in scalable mode.

As a container platform?
We using Monolytic mode to deploy in single Docker container, like this - Install Loki with Docker or Docker Compose | Grafana Loki documentation

But its in cluster of 3 nodes (for redunduncy), like this

P.S. - Now its impossible for us to deploy in k8s or any other orchestration plarform, because of inner management problems :slight_smile:
As a result - we got 3 compactors and its conflicting sometimes. So thats why i am asking about mechanism to exclude compactor, because i want to deploy compactor only once, like this:

I don’t believe you can single out compactor. You can see a list of all components here Loki components | Grafana Loki documentation, and several aliases such as read and write. The problem is the moment you deviate from using just all then you have to go with either simple scalable mode or microservice mode.

My understanding is that in a cluster setting, multiple compactors will form a ring membership and elect one amongst themselves to run compaction. I suspect your issue is that you are not really running your 3 Loki instances as a cluster, instead they are all just individual Loki instances with identical configuration but they aren’t really forming a cluster.

I would recommend you to stand up a dev cluster and configure ring membership, and see if you can get your separate Loki containers clustered. They would probably solve your issues.