I’ve configured promtail to apply labels based on a regex pattern for a Python application (not mine). It works well for properly formatted lines, but it doesn’t work well for unhandled Python exceptions. Is there a way to capture multiple lines as part of the message with the regex parser?
As an example, here are 2 log lines. I’d like the second line to include the entire traceback in its message. As it is now, this would record 8 separate log lines, with the last 6 having no labels.
2.7.1.0-b1758::2023-11-22 09:42:07,791::INFO::[_cplogging:201]::[SNMPManager]::[22/Nov/2023:09:42:07] ENGINE Stopped thread 'Audit Logger'.
2.7.1.0-b1758::2023-11-22 09:42:07,792::ERROR::[_cplogging:201]::[SNMPManager]::[22/Nov/2023:09:42:07] ENGINE Error in 'stop' listener <bound method SNMPManager.stop_monitor of <serv.bin.SNMPManager.SNMPManager object at 0x7f2c77fc3a10>>
Traceback (most recent call last):
File "build/bdist.linux-x86_64/egg/lib/cherrypy/process/wspbus.py", line 197, in publish
File "build/bdist.linux-x86_64/egg/serv/bin/SNMPManager.py", line 113, in stop_monitor
File "/usr/lib64/python2.7/threading.py", line 942, in join
raise RuntimeError("cannot join current thread")
RuntimeError: cannot join current thread
This is the pipeline I’m using:
pipeline_stages:
- regex:
expression: '^(?P<version>.+)::(?P<time>.+)::(?P<level>.+)::\[(?P<source>.+)\]::\[(?P<component>.{1,16}).*\]::(?P<message>.+)$'
- labels:
version:
time:
level:
source:
component:
- output:
source: message
