Promtail - only get/process the latest log after Promtail is started

[bryantlin]

Hi, I am pretty new to Loki and Promtail and tried to find if there was similar posts but can’t find one.

I am trying to find how to let Promtail only process the “new” change after Promtail is running. For example, Promtail started at 2 am and it only processes any changes to log file (no matter if it’s a new or existing file) since 2 am.

The reason why I am asking is because there are many existing large log files on server and if Promtail processes all of those OLD log information, the performance of the server will be affected.

Look forward to having some guidance from you. Thanks in advance.

[wlargou]

Hi,

By default promtail scraps all logs, if you have your old logs in separate files you can exclude them using promtail match stage. If you have the same log i would suggest you opt for rotating your logs which is more efficient and less resources intensive.

Hope it helps.

Good Luck

[bryantlin]

Thanks wlargou