I have configured my cluster to send data to grafana cloud to evaluate if that is an option for my production cluster.
Of course this is not a fresh cluster, so there are already lots of logs in kubernetes available and promtail is complaining in the logs that the rate limit is exceeded.
This leads me to two questions:
- Wouldn’t it make sense to have less restrictive rate ölimiting for a while after the account creation to give promtail a chance to catch up?
- Will promtail recover and just transmit the logs slowly until everything is transfered or will it stay in error and I have to find another solution?