Hello!
I’ve configured a promtail-loki stack to push service logs from a test machine. For context, I have multiple services running in Docker (10+ services) all logging to a volume called Logs, in their respective subfolders (e.g /Logs/service1, /Logs/service2). I have promtail pushing the contents of that entire volume to Loki, and then I’m separating them in Grafana by regex on the filename value. That all works great.
Now I’m trying to do some more parsing, but running into a situation where some of the services are using a different logging format, so they get left out or incorrectly parsed when parsing with ‘pattern’ or ‘regex’.
I’ve thought about excluding those using something like
- source_labels: [__path__]
regex: /logs/(service4|service5|service6)/.*\.log
action: drop
in Promtail, and then configuring a separate job for them, but then I kinda lose on the “service discovery” aspect of the solution and it makes visualization in grafana a little messy (if I want to use a single panel structure and change it based on variables, like I do now). Just wondering if there’s maybe a better solution out there.
If the explanation isn’t clear I can elaborate further, also if this was already discussed/solved somewhere please redirect me to the topic, thanks.