Parsing extended apache logs

Grafana Loki
,
1

I have this apache config line:

LogFormat "%h %l %u %t \"%r\" %>s %I %O %^FB %D \"%{Referer}i\" \"%{User-Agent}i\"" combined_io

which becomes something like:

127.0.0.1 - - [26/Jul/2025:00:09:14 +0200] "GET /server-status/?auto HTTP/1.1" 200 147 833 483 829 "-" "Prometheus-Apache-Exporter/1.0.9"

and I have this promtail config:

scrape_configs:
- job_name: apache2
  pipeline_stages:
    - regex:
        expression: "^(?P<client>(?:[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}|(?:(?:[0-9a-fA-F]{1,4}:){7}[0-9a-fA-F]{1,4}|(?:(?:[0-9a-fA-F]{1,4}:)*[0-9a-fA-F]{1,4})?::(?:[0-9a-fA-F]{1,4}:)*[0-9a-fA-F]{1,4}))) (?P<logname>(?:[a-zA-Z0-9_]+|-)) (?P<user>(?:[a-zA-Z0-9_]+|-)) \\[[^]]+\\] \"(?:(?P<method>.*) (?P<path>[^?]*)(?:\\?[^ ]*)? (?P<proto>.*)/(?P<proto_version>(?:[1-9][0-9]*|0).(?:[1-9][0-9]*|0))|(?P<trash>.*))\" (?P<status>(?:[1-9][0-9]*|0)) (?P<received>(?:[1-9][0-9]*|0)) (?P<sent>(?:[1-9][0-9]*|0)) (?P<ttfb>(?:(?:[1-9][0-9]*|0)|-)) (?P<request_time_us>(?:[1-9][0-9]*|0)) (?P<referer>\"(.*)\") (?P<user_agent>\"(.*)\")$"
      labels:
        client:
        logname:
        user:
        method:
        path:
        status:
        received:
        sent:
        ttfb:
        response_time:
        referer:
        user_agent:

I can check with promtail --inspect that the lines are properly parsed:

mdione@diablo:~$ cat /var/log/apache2/access.log | promtail --stdin --config.file /etc/promtail/config.yml --inspect --dry-run
[inspect: regex stage]:
{stages.Entry}.Extracted["client"]:
        +: 127.0.0.1
{stages.Entry}.Extracted["logname"]:
        +: -
{stages.Entry}.Extracted["method"]:
        +: GET
{stages.Entry}.Extracted["path"]:
        +: /server-status/
{stages.Entry}.Extracted["proto"]:
        +: HTTP
{stages.Entry}.Extracted["proto_version"]:
        +: 1.1
{stages.Entry}.Extracted["received"]:
        +: 147
{stages.Entry}.Extracted["referer"]:
        +: "-"
{stages.Entry}.Extracted["request_time_us"]:
        +: 829
{stages.Entry}.Extracted["sent"]:
        +: 833
{stages.Entry}.Extracted["status"]:
        +: 200
{stages.Entry}.Extracted["trash"]:
        +: 
{stages.Entry}.Extracted["ttfb"]:
        +: 483
{stages.Entry}.Extracted["user"]:
        +: -
{stages.Entry}.Extracted["user_agent"]:
        +: "Prometheus-Apache-Exporter/1.0.9"
2025-07-26T19:18:52.78860809+0200       {__path__="/var/log/apache2/access.log", job="apache2_logs"}    127.0.0.1 - - [26/Jul/2025:00:09:14 +0200] "GET /server-status/?auto HTTP/1.1" 200 147 833 483 829 "-" "Prometheus-Apache-Exporter/1.0.9"

I don’t know how to check on loki, but on grafana only see the full lines and no breakdown:

image
image1920×660 66.4 KB

I don’t know what am I missing. Any hints?

2

My recommendation to you would be:

  1. Parse the logs in promtail for timestamp only and nothing else.
  2. In Loki, you can then use pattern filter when querying the logs.