The cloud logs export feature is great, but it seems like a major limitation is the inability to split logs by labels.
One of the bigger reasons you may want to do this is to apply different retention rules for different labels (e.g. environments/stages).
It seems like another option would be to use multiple Grafana Stacks, but then you end up having to manage users across different stacks. Additionally, users would have to visit different sites just to query logs across environments.
Are their any best practices for how to accomplish the desired goal of segmenting exported logs by labels AND have a single pane of glass for users?
You can have different retention rules for different log streams. See Retention | Grafana Loki documentation.
I am not sure I understand the rest of your question. Can you post some screenshots and perhaps a bit more details on what you are trying to accomplish?
@tonyswumac I’m referring to Cloud Logs Export | Grafana Cloud documentation.
The retention I’m speaking of is that on cloud storage like S3.
When setting up the cloud exports feature, you can only select a single S3 bucket and path, with no way to segment by Loki labels.
What I’m asking for is a way to provision multiple S3 buckets, each exporting logs filtered by Loki labels.
I know this is not possible today, but hoping there’s a way around it.
Ah, ok. I don’t think I have a satisfactory answer or this, but Loki’s API is pretty easy to use, and it should be pretty straight forward to write a script that exports logs from Loki over a period of time with intervals and with a set of labels.
