I Had deployed a test cluster with 3 writers and 3 readers .
However i’m just thinking if is possible in microservice mode to have separate clusters.
Example:
Write Path:
ingester 10 pods
distributor 10 pods
Controll path:
index-gateway
ruler
compactor
Read Path small team
query-frontend x 3 pods
query-scheduler x3 pods
querier x3 pods
Read Path Big team
query-frontend 15pods
query-scheduler 15 pods
querier 15 pods
I’m looking to create a big ingestion cluster however base by the team requirements create small clusters for part as they my not need a lot of data … however other teams my be focused on more data and they need more resources on read.
Maybe some other way that this can be done …
Also your readers need connectivity to all writers, too, so if it’s not within the same cluster you’ll probably have to do a lot of stitching.
Team requirement shouldn’t change your approach really. If they are using the same Loki cluster, it doesn’t matter if one person hits your reader or 1000 people hit it, you should just put in some sort of reliable scaling method and scale your readers up and down accordingly.
My need of this is for older data … let’s say … i have a team that is doing incident audit … this means it will always look on older that is already sorted on s3. Like … there will be another loki cluster that will connect to the same bucket but always search older data than 10 days … however it will do huge searches so this will require big cluster that my try corelate multiple tenants .
I guess I still don’t fully grasp the necessity to have a different cluster, searching logs older than 10 days is no different from search logs that’s older than 10 seconds. But I digress.
I haven’t done this before, but you should be able to have another set of readers, for example in your ring membership you might have:
writers.local
readers.local
archive-readers.local
And you’ll have another set of query frontend as well, but you’ll want to make sure you have a different configuration for the second read path to disable the compactor.
Alternatively, you should be able to create an entirely separate Loki cluster with the same storage backend configuration. You’ll want to make sure the writer path for that separate cluster is blocked off, and again make sure to disable compactor.
