How do I push logs from docker_loki_driver to loki_push_api?

Grafana Loki
, , ,
1

I have configured my promtail scrape config like this:

scrape_configs:
  - job_name: docker
    loki_push_api:
      server:
        http_listen_port: 1515
        log_level: "debug"
      labels:
        job: "dockerjson"
    pipeline_stages:
      - json:
          expressions:
            output: log
            stream: stream
            timestamp: time
            context: context

And I am using a test docker compose container that only prints one log line and pushes that line to the loki_push_api from promtail.

services:
  test-log:
    image: ubuntu:22.04
    container_name: "test-logger"
    hostname: 'test-logger'
    command: 'echo {"context":"ConfigService","level":"info","message":"Loading JWT Keys","timestamp":"2024-05-02T14:00:00.008Z"}'
    logging:
      driver: loki
      options:
        loki-url: "http://10.64.194.113:1515/loki/api/v1/push"
        loki-external-labels: "container_name={{.Name}}"

Then I get the error in promtail:
level=debug ts=2024-05-07T07:22:36.476108292Z caller=json.go:145 component=push_pipeline_docker component=stage type=json msg="failed to unmarshal log line" err="ReadString: expects \" or n, but found c, error found in #2 byte of ...|{context:Con|..., bigger context ...|{context:ConfigService,level:info,message:Loading JW|..."

Is there a way to debug it with --dry-run and --stdin because I use the push api server and that gets the message from the docker log driver?

1 Like
2

I solved it like this:

docker-compose.yml for example docker driver configuration:

services:
  test-log:
    image: ubuntu:22.04
    container_name: "test-logger"
    hostname: 'test-logger'
    command: echo '{"context":"ConfigService","level":"info","message":"Loading JWT Keys","timestamp":"2024-05-07T09:10:00.008Z"}'
    logging:
      driver: loki
      options:
        loki-url: "http://10.64.194.113:1515/loki/api/v1/push"
        loki-external-labels: "container_name={{.Name}}"
        loki-pipeline-stages: |
          - json:
              expressions:
                context: context
                level: level
                message: message
                timestamp: timestamp
          - labels:
              context:
              level:
              message:
              timestamp:

And my promtail scrape-config is now very simple and does nothing except parse the timestamp:

scrape_configs:
  - job_name: docker
    loki_push_api:
      server:
        http_listen_port: 1515
        log_level: "debug"
      labels:
        job: "dockerjson"
    pipeline_stages:
       - timestamp:
           source: timestamp
           format: RFC3339