Losing logs after increasing log cardinality limit

Grafana Loki
,
1

I usually get an error upon querying for logs “limit exceeded for logs{app}; 102799 entries, more than limit of 100000”

But when I increase the cardinality limit to say 10x of the existing number I start losing logs,
or start to see gaps in logs when checked in grafana. Is there something I should be mindful of ?

I have deployed loki as loki-scalabale deployment

2

Are you querying from a browser or API call?

I find that in general you don’t need too many log lines returned in a browser, because you likely aren’t going to be browsing through 100K lines of log in a browser window. If you are doing API call, it may be a good idea to query in a smaller time frame and to it in loops.