Does Loki perform bad when cardinality of the 'filename' label is high

[kafenut]

In my case, some logs are named like this xxxxx.2023050401.log.
Our application will create a new log file every hour. Eventually, the “filename” label becomes a dynamic label with unbounded values.

[tonyswumac]

You have two options:

1. Remove the filename label on ingestion, if you don’t care about knowing the file name.

2. Parse the file name, and extract the xxxxx part, and use that as file name (essentially get rid of the date part).

[etiennedesclaux]

@tonyswumac

Remove the filename label on ingestion

Is there a way to remove the filename label in the Promtail config?

How can I remove it?

[tonyswumac]

Since filename is a label, you can drop it using labeldrop, see labeldrop | Grafana Loki documentation.

If you wish to preserve it, you can parse the label and then overwrite it like so (not tested):

regex:
  source: "filename"
  expression: "^(?P<new_filename>[^\.]+)\.\S+$"

labels:
  filename: "new_filename"

[b0b]

A small note that I found when I was trying to remove the filename label my self…

From https://github.com/grafana/loki/blob/0fc934e4ea3ed76b995fd488a5fec3fab5a82886/docs/sources/clients/promtail/scraping.md?plain=1#L63-L65

The label `filename` is added for every file found in `__path__` to ensure the uniqueness of the streams. It is set to the absolute path of the file the line was read from.

So at least I, in the end, decided against removing the filename label.