"line_format" reduces log count to only a few

Hello, I’ve been using Loki for a bit and noticed weird behaviour when line_format is applied to the log query.

We have exactly 90 (testing) logs for a given query

{env=`stage`,region=`REGION`, cluster=`CLUSTER_NAME`, namespace=`NS`, app=`APP`}

And everything shows fine, however problem happens when line_format is used ("{{.}}" is just for testing to see all fields).

{env=`stage`,region=`REGION`, cluster=`CLUSTER_NAME`, namespace=`NS`, app=`APP`} | line_format "{{.}}"

Suddenly amount of logs shown are heavily reduced from 90 to 1, sometimes 2 (“Log volume” graph is still correct).

Which is quite confusing since query itself show all logs until line_format is used.

Screenshot #1 - no line_format

Screenshot #2 - with line_format

I wonder if I’m missing something, searched over the internet but could not find relevant info, so looks like a bug, appreciate any help/assistance.

Unfortunately can’t confirm issue was not happening before upgrade to Loki v3.0.0

Loki details

Loki version: 3.0.0
Helm chart: 6.2.0
Deployment: Simple Scalable

I ran into a similar issue and was able to collect a bit more information, specifically comparing to an older version 2.7.5. I’ve made a github issue Loki v3.0.0 quietly gives incomplete results for some queries including a line_format · Issue #12798 · grafana/loki · GitHub and posted in the community slack Slack

1 Like