I have deployed promtail and loki separately using helm.
I observed the following error in the promtail logs
level=warn ts=2024-05-30T07:02:56.747829793Z caller=client.go:419 component=client host=hw-loki.monitoring.svc:3100 msg="error sending batch, will retry" status=429 tenant= error="server returned HTTP status 429 Too Many Requests (429): Ingestion rate limit exceeded for user fake (limit: 4194304 bytes/sec) while attempting to ingest '50527' lines totaling '1048502' bytes, reduce log volume or contact your Loki administrator to see if the limit can be increased"
Will this cause my loki to not collect data and how should I change my loki configuration?
Yes, this means your logs are being dropped. There are several configurations you can tweak to get resolve this error, see Request Validation and Rate-Limit Errors | Grafana Loki documentation…
I’m using monolithic mode, assuming I improve the deployment mode (Simple Scalable Deployment or Microservices mode) without modifying the parameters, will it be an improvement on the error I’m getting now?
Tweaking the parameters will remove the error, but whether your monolithic deployment can handle it is a different question.
You’d still need to tweak the parameter if you go simple scalable mode. But in general I’d recommend at least considering it if you have reasonable amount of log volume.
