Loki multi tenancy - Can it be used for large SaaS application?

Grafana Loki
1

Our product comprises a collection of microservices catering to a substantial customer base. Currently, we are in the process of assessing Grafana Loki as a logging solution. Within our application logs, the tenantId is a crucial component that I am extracting and establishing as the tenant identifier. It has come to my attention that Loki inherently supports multi-tenancy. However, I am uncertain about its feasibility in our scenario, considering the potential translation of this feature into managing thousands of tenants. I seek guidance on the most advisable approach for configuration:

  1. Employ Loki’s multi-tenancy functionality, segregating each customer’s logs into distinct tenants. Is this setup scalable given the volume of tenants?
  2. Opt for a single tenant setup, utilizing the pack/unpack/structured metadata feature to affix the tenant context alongside logs and subsequently filtering based on this context during queries. Would this method be more efficient?
  3. Are there alternative approaches recommended for our use case that might offer better scalability and efficiency?

I appreciate any insights or recommendations you can provide regarding the optimal setup for accommodating our extensive customer base within the framework of Loki’s capabilities.

2

In general it’s a good idea to enable multi-tenancy for any reasonable amount of log volume, but whether to line that up with your application’s tenant ID would be decided by whether your logs are made available to your customers or not (at least in my opinion). If you have to make your logs available to your customers, then having a Loki tenant for each of your customers makes logistical sense, and will make your access control much easier.

If you don’t need to make your logs available to your customer, then I don’t think you necessarily want to split your logs per tenant (it might make querying and aggregation more difficult). I would make the tenant ID a label, and you can then filter / aggregate your logs by the tenant label.

3

Thank you. This helps.