Hello , I face with a permanent crashing of prom tail’s container with an error which is “failed to make file target manager: too many open files”. I use official docker compose file.
Additionally I added ulimits parameter , but it didn’t help .
version: “3”
networks:
loki:
services:
promtail:
image: grafana/promtail:2.8.0
volumes:
- /var/log:/var/log
command: -config.file=/etc/promtail/config.yml
ulimits:
nofile:
soft: 65536
hard: 65536
networks:
- loki
Error from a container:
level=error ts=2023-04-20T21:50:49.989724713Z caller=main.go:170 msg=“error creating promtail” error=“failed to make file target manager: too many open files”
Others have run into this too – the general advice is that upping the number of open files allowed with ulimit tooling is the only real option. I did find a related thread though that indicated usually this is caused by another process on the same machine, not promtail itself. In other words, promtail here may be the victim not the culprit.
Given that you’re running docker compose, you might want to check your host OS and how it’s doing, just as a thought. Your options boil down to increasing the limit with ulimit or terminating some other fd-hungry process.
