Serilog compact json formatter not being parsed

[djefobey]

I am running a basic install of loki-stack helm chart (Loki, Promtail, Grafana, Prometheus) with Asp.Net core services logging with serilog in json format (compactJsonFormatter), however, when I check in loki the logs aren’t parsed correctly:

I am not doing any custom pipelines, should I add a pipeline and if so, any examples?

[b0b]

Hello,

I’m in a similar situation. I’m just starting to research Promtail pipeline_stages.

I would not expect those logs to just automagically be parsed by Loki. Looks like you have a mixed log format as well, which starts with a string and then there is a JSON object at the end.

I come from Elastic Stack and my expectation is that logs have to be parsed before they are stored but that is not necessarily true for Loki. You could parse your logs in Grafana if you want with LogQL.

First use pattern to get the JSON object separated. This you can pipe to json.

Here is an example of a LogQL query I have used to do something similar

{container="my-container"} | pattern "<time>\t<level>\t<component>\t<status>\t<json_log>\n" | line_format "{{.json_log}}"

You will have to adjust the pattern to your own use case.

What I’m trying to figure out for myself is, how much should I parse my logs before ingestion and what should I leave for Grafana and LogQL.