How to render a log with `level=DEBU` in grafana?

[Zwei]

Expecte

Here are some logs:

2023-03-20 13:17:21.625 [INFO] (?P<msg>)
2023-03-20 13:17:21.625 [INFO] {08762e5dd8084e176b6780683d9ae896} (?P<msg>)
2023-03-20 13:17:21.629 [DEBU] {6471af92d9084e178dfedd11c98d743d} (?P<msg>)

I hope to see level=debug in grafana instead of unknown

What I did

I checked the related issues: Loki does not recognize log level `ERRO` · Issue #7944 · grafana/loki, Promtail: replace stage can`t replace GUID · Issue #8344 · grafana/loki

I try to add a stage , but it didn’t work:

- replace:
    expression: "(?P<level>DEBU)"
    source: level
    replace: "debug"
- replace:
    expression: "(?P<level>ERRO)"
    source: level
    replace: "error"    

ENV

promtail

• version: 2.7.3
• OS/Arch: window 10/amd64

loki

• docker image: grafana/loki:2.7.3
• docker version: 23.0.1
• docker OS/Arch: linux/amd64

More info

promtail config.yml :

scrape_configs:

  - job_name: job
    static_configs:
    - labels:
        __path__: /var/log

    pipeline_stages:

    - multiline:
        firstline: '^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}.\d{3}'
        max_wait_time: 3s
    - regex:
        expression: '(?P<time>^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}.\d{3}) \[(?P<level>[A-Z]+)\]( {(?P<trace>\w*)}){0,1} (?P<msg>(.|\n)*$)'

    - labels:
        time:
        level:
        trace:

    - replace:
        expression: "(?P<level>DEBU)"
        source: level
        replace: "debug"
    - replace:
        expression: "(?P<level>ERRO)"
        source: level
        replace: "error"    
    
    - timestamp:
        source: time
        format: '2006-01-02 15:04:05.000'
        
    - output:
        source: msg

Thanks in advance.

[tonyswumac]

Maybe try "DEBU" and "ERRO" as your regex expression?

[Zwei]

2023-03-21 09:34:53.998 [INFO] {244d0c33494b4e174b2e014e3b119b21} swagger ui is serving at address: http://127.0.0.1:8180/swagger/
2023-03-21 09:34:54.001 [DEBU] {f42c68674a4b4e178559bb682711750b} [ 11 ms] [default] [exam_dev_pi] [rows:1  ] SELECT * FROM

I can modify label msg:

    - replace:
        expression: '(default)'
        source: msg
        replace: "~MySQL~"

    - replace:
        expression: '(http://127.0.0.1)'
        source: msg
        replace: "http://192.168.32.132"

however, in label level, I fail

    - replace:
        expression: "[(?P<level>DEBU)]"
        source: level
        replace: "debug"

    - replace:
        expression: "DEBU"
        source: level
        replace: "debug"

    - replace:
        expression: "(DEBU)"
        source: level
        replace: "debug"

[Zwei]

My answer: using replace stage before regex stage

server:
  http_listen_port: 9080
  grpc_listen_port: 0
 
positions:
  filename: /var/promtail/positions.yml
 
clients:
  - url: http://192.168.32.71:3100/loki/api/v1/push

scrape_configs:
  - job_name: 132-exam
    static_configs:
    - labels:
        job: 132-exam
        dir: app
        __path__: /var/log/*.log
        
    pipeline_stages:
    - multiline:
        firstline: '^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}.\d{3}'
        max_wait_time: 3s
    
    - replace:
        expression: '(\[DEBU\])'
        replace: "[DEBUG]"

    - replace:
        expression: '(\[ERRO\])'
        replace: "[ERROR]"

    - regex:
        expression: '(?P<time>^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}.\d{3}) \[(?P<level>[A-Z]+)\](?:\s+{(?P<trace>\w*)})? (?P<msg>(.|\n)*$)'

    - labels:
        time:
        level:
        trace:

    - timestamp:
        source: time
        format: '2006-01-02 15:04:05.000'
        location: "Asia/Shanghai"

    - replace:
        expression: '(http://127.0.0.1)'
        source: msg
        replace: "http://192.168.32.132"

    - output:
        source: msg