We just upgraded to latest versions of both loki and promtail (v.2.2.0), mainly to have the multiline feature on stacktrace (exceptions) appearing nicely in loki/grafana and in one log block. However by testing the feature, we noticed two issues:
config example:
snippets:
pipelineStages:
- cri: {}
- multiline:
firstline: '^\d{4}-\d{2}-\d{2} \d{1,2}:\d{2}:\d{2}.\d{3}'
the regex matches the start of the stacktrace correctly:
2021-03-25 09:26:07.499 [-][-][-][ INFO] [,]...
- while the first block with the timestamp appears as one log line in Loki/Grafana, every line of stack trace is next to each other instead of a new line, thus making really hard to read it. Is there way to make every line of stack trace to appear in a better format ie more readable?
- the “caused by” block of the same exception stacktrace appear as separate log line in Loki/grafana, even though is shouldn’t since it belongs to the previous block.
FYI, everything else is kept as default for the multiline: e.g. max_lines: 128 and max_wait_time: 3s.
regarding the second issue, 128 lines haven’t been exceeded to cause this.
Thank you for your work!