Odoo logs in loki/promtail

Hello, i have odoo logs which is not in logfmt format. I run all my apps in docker and i have added labels to each service i want to get logs from these labels:

labels:
      - logging=promtail
      - logging_jobname=containerlogs

Logs look like this.


odoo-1   | 2024-02-02 06:57:12,670 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:12] "POST /mail/load_message_failures HTTP/1.0" 200 - 3 0.003 0.007
odoo-1   | 2024-02-02 06:57:12,671 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:12] "POST /web/action/load HTTP/1.0" 200 - 9 0.016 0.050
odoo-1   | 2024-02-02 06:57:12,704 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:12] "GET /websocket HTTP/1.0" 400 - 1 0.001 0.001
odoo-1   | 2024-02-02 06:57:12,719 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:12] "POST /web/dataset/call_kw/board.board/get_views HTTP/1.0" 200 - 22 0.012 0.011
odoo-1   | 2024-02-02 06:57:12,762 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:12] "POST /web/action/load HTTP/1.0" 200 - 9 0.008 0.011
odoo-1   | 2024-02-02 06:57:12,768 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:12] "POST /web/action/load HTTP/1.0" 200 - 9 0.013 0.012
odoo-1   | 2024-02-02 06:57:12,773 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:12] "POST /web/action/load HTTP/1.0" 200 - 8 0.020 0.008
odoo-1   | 2024-02-02 06:57:12,782 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:12] "POST /web/action/load HTTP/1.0" 200 - 8 0.026 0.010
odoo-1   | 2024-02-02 06:57:12,808 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:12] "POST /web/dataset/call_kw/fms.flock/get_views HTTP/1.0" 200 - 5 0.011 0.014
odoo-1   | 2024-02-02 06:57:12,828 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:12] "POST /web/dataset/call_kw/fms.animal/get_views HTTP/1.0" 200 - 19 0.031 0.023
odoo-1   | 2024-02-02 06:57:12,831 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:12] "POST /web/dataset/call_kw/fms.animal/get_views HTTP/1.0" 200 - 16 0.031 0.022
odoo-1   | 2024-02-02 06:57:12,844 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:12] "POST /web/dataset/call_kw/fms.flock/web_read_group HTTP/1.0" 200 - 8 0.007 0.009
odoo-1   | 2024-02-02 06:57:12,858 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:12] "POST /web/dataset/call_kw/fms.holding/get_views HTTP/1.0" 200 - 15 0.025 0.034
odoo-1   | 2024-02-02 06:57:12,865 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:12] "GET /web/bundle/web.chartjs_lib?lang=en_US&debug=assets HTTP/1.0" 200 - 2 0.002 0.005
odoo-1   | 2024-02-02 06:57:12,885 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:12] "GET /web/assets/debug/web.chartjs_lib.js HTTP/1.0" 200 - 4 0.005 0.003
odoo-1   | 2024-02-02 06:57:13,135 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:13] "POST /web/dataset/call_kw/fms.animal/read_group HTTP/1.0" 200 - 7 0.282 0.006
odoo-1   | 2024-02-02 06:57:13,143 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:13] "POST /web/dataset/call_kw/fms.animal/web_read_group HTTP/1.0" 200 - 3 0.281 0.009
odoo-1   | 2024-02-02 06:57:13,187 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:13] "POST /web/dataset/call_kw/fms.holding/web_search_read HTTP/1.0" 200 - 17 0.277 0.037
odoo-1   | 2024-02-02 06:57:13,233 1 INFO ? werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:13] "GET /base/static/img/avatar.png HTTP/1.0" 200 - 0 0.000 0.001
odoo-1   | 2024-02-02 06:57:13,592 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:13] "POST /web/dataset/call_kw/fms.animal/read_group HTTP/1.0" 200 - 3 0.401 0.036
odoo-1   | 2024-02-02 06:57:13,610 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:13] "POST /web/dataset/call_kw/fms.animal/read_group HTTP/1.0" 200 - 2 0.452 0.008
odoo-1   | 2024-02-02 06:57:13,689 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:13] "POST /web/dataset/call_kw/fms.animal/read_group HTTP/1.0" 200 - 7 0.499 0.037
odoo-1   | 2024-02-02 06:57:13,765 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:13] "POST /web/dataset/call_kw/fms.animal/read_group HTTP/1.0" 200 - 3 0.573 0.031
odoo-1   | 2024-02-02 06:57:14,201 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:14] "POST /web/dataset/call_kw/fms.animal/read_group HTTP/1.0" 200 - 8 1.007 0.034
odoo-1   | 2024-02-02 06:57:14,228 1 INFO ? werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:14] "GET /board/static/img/layout_1-2.png HTTP/1.0" 304 - 0 0.000 0.003
odoo-1   | 2024-02-02 06:57:14,685 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:14] "GET /websocket HTTP/1.0" 400 - 1 0.001 0.004
odoo-1   | 2024-02-02 06:57:15,930 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:15] "GET /websocket HTTP/1.0" 400 - 1 0.002 0.005
odoo-1   | 2024-02-02 06:57:19,991 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:19] "GET /websocket HTTP/1.0" 400 - 1 0.002 0.005
odoo-1   | 2024-02-02 06:57:21,492 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:21] "POST /web/action/load HTTP/1.0" 200 - 9 0.011 0.022
odoo-1   | 2024-02-02 06:57:21,660 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:21] "POST /web/dataset/call_kw/fms.animal/get_views HTTP/1.0" 200 - 78 0.037 0.100
odoo-1   | 2024-02-02 06:57:21,698 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:21] "POST /web/dataset/call_kw/res.users/has_group HTTP/1.0" 200 - 2 0.002 0.004
odoo-1   | 2024-02-02 06:57:21,999 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:21] "POST /web/dataset/call_kw/fms.animal/web_read_group HTTP/1.0" 200 - 3 0.300 0.007
odoo-1   | 2024-02-02 06:57:22,125 1 INFO ? werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:22] "GET /web/static/lib/odoo_ui_icons/fonts/odoo_ui_icons.woff2 HTTP/1.0" 200 - 0 0.000 0.003
odoo-1   | 2024-02-02 06:57:23,735 1 INFO fms17 werkzeug: 173.8.0.4 - - [02/Feb/2024 06:57:23] "GET /websocket HTTP/1.0" 400 - 1 0.002 0.005

I tried to create dashboard for logs but could not even get method POST/GET out of these lines It’s not in logfmt format. I could not figure out how can i get this information in promtail. can i reformat logs to get key/value pairs? I wanted to build dashboard like in this video https://www.youtube.com/watch?v=EPLvB1eVJJk

Thank you in advance for helping. I appreciate it.

You can either parse the logs on promtail or parse the logs on Loki. If you want POST/GET as a label then you’ll have to do it in promtail (most likely with a regex filter), but personally I prefer to keep log pipeline simple and parse in Loki.

Given your logline, here is an example of how you might parse it:

{<SELECTOR>} | pattern `<_> | <_> <_> <_> <log_level> <_> <_> <IP> <_> <_> [<_> <_>] "<method> <url> <http_version>" <return_code> <_> <_> <_> <_>`

See LogQL Analyzer | Grafana Loki documentation

The example I gave you is a LogQL query, that’s for you to parse after logs are sent to Loki. If you want to do it from promtail you’ll need to use regex parsing combined with labels. Here is an example from another thread: Use loki docker plugin to label python logger - #3 by lekspek

You should also be mindful of what you use as labels. See Label best practices | Grafana Loki documentation

In Grafana Explorer, on the top right, select “code” view, and then you should see a place for you to enter the query. I’d also suggest reading through LogQL: Log query language | Grafana Loki documentation if you haven’t.

If a query is working properly each of the field parsed gets turned into a label at run time.