Use loki docker plugin to label python logger

Grafana Loki
, , , ,
1

My goal: Label the python logger from docker containers, and also keep the docker container label.

What I did and what works for now:
I have a working Loki instance and installed Loki as a docker plugin on the local environment.
Here is what I’ve added to my daemon.json:

  "debug" : true,
  "log-driver": "loki",
  "log-opts": {
     "loki-url": "http://loki.local/loki/api/v1/push",
     "loki-batch-size": "400"
    }

I can see logs in Loki :grinning:, and they look like that:

[INFO][2023-08-15T16:23:22.856312+03:00][MainThread]: RB operation has been completed. The next operation is schedule to start in 3600 seconds. (main_rb.py:179)
[INFO][2023-08-15T16:23:22.856201+03:00][MainThread]: No jobs were found in db (main_rb.py:70)
[INFO][2023-08-15T16:23:22.854520+03:00][MainThread]: Wake-up and run RB operation... (main_rb.py:177)

each log have container_name, container_id, stack_name (those are enough for me).

What missing: I want to parse the severity of the log level ( INFO,DEBUG,WARNING,ERROR,FATAL ), Thread (we can see MainThread on this log, but can be something else), and the python file (in this example it’s main_rb.py)

What I’ve tried: Found this article: Configuration | Grafana Loki documentation, but not sure how to work with loki-pipeline-stages on JSON, and what regex is good for me.
What is the best practice here? what is the regex that can fit here?

2

The regexp is probably your best bet. Something like:

| regexp `^\[(?P<log_level>\S[^\[]+)\]\[(?:\S[^\[]+)\]\[(?P<thread>\S[^\[]+)\].+\((?P<file_name>\S+\.\S+):`

You can also try it here: LogQL Analyzer | Grafana Loki documentation

3

This answer is accepted even if the regex won’t work. because I wasn’t familiar with the simulator page and it’s very helpful.
Hope someone can help in the future to know to use daemon.json properly.

Edit:
The regex is working well :nerd_face:
For anyone who’s going to face that, I’ve added config.yaml inside /var/lib/docker/plugins/0dc182b0a98f3689ea8d5c74d9ce568e698a342d3ca6aecf6411b18c7d6a8e14/rootfs/home/config.yaml (you might need replace 0dc182b0a98f3689ea8d5c74d9ce568e698a342d3ca6aecf6411b18c7d6a8e14 with your plugin ID)
and it’s looking like that now:

pipeline_stages:
- regex:
     expression: '^\[(?P<log_level>\S[^\[]+)\]\[(?:\S[^\[]+)\]\[(?P<thread>\S[^\[]+)\].+\((?P<file_name>\S+\.\S+):'
- labels:
    log_level:
    thread:
    file_name:

and added loki-pipeline-stage-file section to my docker daemon (/etc/docker/daemon.json):

"debug" : true,
  "log-driver": "loki",
  "log-opts": {
     "loki-url": "http://my-loki.local/loki/api/v1/push",
     "loki-batch-size": "400",
     "loki-pipeline-stage-file":"/home/config.yaml"
    }

Disable & Enable your plugin to make sure it’s working.