Loki driver docker no logs

Grafana Loki
1

Hello everyone,

I’ve been driven mad for a week now with an issue related to the Loki driver for Docker.

I’ve followed the simple installation documentation. However, my Docker system logs never seem to make it to Loki. Interestingly, my application logs are successfully sent via Promtail.

The Loki server is definitely reachable from the containers.

The logs no longer show up when using docker logs, which seems to confirm that they are indeed being sent somewhere.

If anyone has any ideas :sweat_smile:, I’ve scoured the internet and consulted ChatGPT to no avail.

I am using Docker Desktop for Mac ARM.

docker version
> Client:
  Version:           24.0.7
  API version:       1.43
  Go version:        go1.20.10
  Git commit:        afdd53b
  Built:             Thu Oct 26 09:04:20 2023
  OS/Arch:           darwin/arm64
  Context:           orbstack
> Server: Docker Engine - Community
  Engine:
    Version:          24.0.7
    API version:      1.43 (minimum version 1.12)
    Go version:       go1.20.10
    Git commit:       311b9ff
    Built:            Thu Oct 26 09:08:17 2023
    OS/Arch:          linux/arm64
    Experimental:     false
  containerd:
    Version:          v1.7.7
    GitCommit:        8c087663b0233f6e6e2f4515cee61d49f14746a8
  runc:
    Version:          1.1.9
    GitCommit:        82f18fe0e44a59034f3e1f45e475fa5636e539aa
  docker-init:
    Version:          0.19.0
    GitCommit:        de40ad0

loki-docker-driver:2.9.4

> docker plugin list
ID             NAME          DESCRIPTION           ENABLED
c5b1f4293b46   loki:latest   Loki Logging Driver   true

The configuration added under my containers in the docker-compose.

yamlCopy code

> logging:
    driver: loki
    options:
      loki-url: "http://loki:3100/loki/api/v1/push"
      loki-external-labels: job=php74_logs,environment=production

Conf loki

auth_enabled: false


server:
  http_listen_port: 3100


common:
  path_prefix: /tmp/loki
  storage:
    filesystem:
      chunks_directory: /tmp/loki/chunks
      rules_directory: /tmp/loki/rules
  
  replication_factor: 1


  ring:
    instance_addr: 127.0.0.1
    kvstore:
      store: inmemory


schema_config:
  configs:
    - from: 2020-09-07
      store: boltdb-shipper
      object_store: filesystem
      schema: v12
      index:
        prefix: loki_index_
        period: 24h
2

What do you mean by Docker system logs?

3

Thanks for your help :slightly_smiling_face:

I talk docker command “docker logs” or “docker compose logs”

I think to the logs it’s send somewhere because it’s not available with docker log command.

4

Ok, I just found out by trying to tcpdump my stack:

from the promtail container, the URL is indeed http://loki:3100/loki/api/v1/push.

But from the docker loki driver, the call coming from my host and not from the stack, the URL is http://127.0.0.1:3100/loki/api/v1/push.

1 Like
5

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.