CPU usage went high in the beginning when Promtail started to run

Hi, I am running Promtail on server which constantly has log files updated.

But I notice that, when I started to run Protmail, CPU went high (between 40% and 80%) for several minutes (around 8 minutes) and then went down (i.e. back to normal). Is there any suggested config key that I should use to avoid high CPU usage in the beginning of running?

I only use the following three configs:

  • batchsize: 10240
  • batchwait: 1s
  • sync_period: 5s

Or, is this situation expected?

Thank you for your advice in advance!

I have noticed this behavior as well. I tested across hosts that had a number of log files existing already (no need to bring up proper log rotation!) and when Promtail starts is reads them all and updates Loki.

Promtail used the CPU on the host until it was done processing the files that existed prior to the Promtail installation.

I had this backoff_config in place

backoff_config:
min_period: 100ms
max_period: 10s
max_retries: 20
batchwait: 2s
batchsize: 819200

I would love a feature that allowed me to update loki with the initial logs slowly over time using a certain amount of resource vs. using all the CPU resources on the machine for as long as it takes until it’s done.