Promtail not pushing logs to loki/grafana cloud

Grafana Cloud
, ,
1

Hello Folks,

Need help with - Logs are not being pushed to Grafana Cloud Loki, here is promtail output:

(base) mac@macs-MacBook-Pro auth % docker logs -f auth-promtail-1

level=info ts=2025-01-14T03:10:02.582657296Z caller=promtail.go:133 msg="Reloading configuration file" md5sum=294099248eea1cf971a3bf2241440903

level=info ts=2025-01-14T03:10:02.583108296Z caller=server.go:322 http=[::]:9080 grpc=[::]:42013 msg="server listening on addresses"

level=info ts=2025-01-14T03:10:02.583300837Z caller=main.go:174 msg="Starting Promtail" version="(version=2.9.0, branch=HEAD, revision=2feb64f69)"

level=warn ts=2025-01-14T03:10:02.583378254Z caller=promtail.go:263 msg="enable watchConfig"

level=info ts=2025-01-14T03:10:07.58523884Z caller=filetargetmanager.go:361 msg="Adding target" key="/app/logs/*.log:{job=\"auth-app\"}"

level=info ts=2025-01-14T03:10:07.588750965Z caller=filetarget.go:313 msg="watching new directory" directory=/app/logs

level=info ts=2025-01-14T03:10:07.589300673Z caller=tailer.go:145 component=tailer msg="tail routine: started" path=/app/logs/auth-app.log

ts=2025-01-14T03:10:07.589470006Z caller=log.go:168 level=info msg="Seeked /app/logs/auth-app.log - &{Offset:0 Whence:0}"

level=info ts=2025-01-14T03:21:21.096513388Z caller=promtail.go:133 msg="Reloading configuration file" md5sum=294099248eea1cf971a3bf2241440903

level=info ts=2025-01-14T03:21:21.096935596Z caller=server.go:322 http=[::]:9080 grpc=[::]:42413 msg="server listening on addresses"

level=info ts=2025-01-14T03:21:21.097025804Z caller=main.go:174 msg="Starting Promtail" version="(version=2.9.0, branch=HEAD, revision=2feb64f69)"

level=warn ts=2025-01-14T03:21:21.097066054Z caller=promtail.go:263 msg="enable watchConfig"

level=info ts=2025-01-14T03:21:26.097119626Z caller=filetargetmanager.go:361 msg="Adding target" key="/app/logs/*.log:{job=\"auth-app\"}"

level=info ts=2025-01-14T03:21:26.098878917Z caller=filetarget.go:313 msg="watching new directory" directory=/app/logs

ts=2025-01-14T03:21:26.099545084Z caller=log.go:168 level=info msg="Seeked /app/logs/auth-app.log - &{Offset:4736 Whence:0}"

level=info ts=2025-01-14T03:21:26.099664917Z caller=tailer.go:145 component=tailer msg="tail routine: started" path=/app/logs/auth-app.log

level=info ts=2025-01-14T03:35:24.475875Z caller=promtail.go:133 msg="Reloading configuration file" md5sum=7331a19877f822e2c711f2885fce6122

level=info ts=2025-01-14T03:35:24.476300708Z caller=server.go:322 http=[::]:9080 grpc=[::]:39125 msg="server listening on addresses"

level=info ts=2025-01-14T03:35:24.476405916Z caller=main.go:174 msg="Starting Promtail" version="(version=2.9.0, branch=HEAD, revision=2feb64f69)"

level=warn ts=2025-01-14T03:35:24.476431875Z caller=promtail.go:263 msg="enable watchConfig"

level=info ts=2025-01-14T03:35:29.478968002Z caller=filetargetmanager.go:361 msg="Adding target" key="/app/logs/*.log:{app=\"auth-app\", env=\"dev\", job=\"auth-app\"}"

level=info ts=2025-01-14T03:35:29.481305002Z caller=filetarget.go:313 msg="watching new directory" directory=/app/logs

level=info ts=2025-01-14T03:35:29.482047419Z caller=tailer.go:145 component=tailer msg="tail routine: started" path=/app/logs/auth-app.log

ts=2025-01-14T03:35:29.482108836Z caller=log.go:168 level=info msg="Seeked /app/logs/auth-app.log - &{Offset:4948 Whence:0}"

Promtail is actively watching the log file (/app/logs/auth-app.log).
It has seeked to the correct offset and started tailing the file.
No errors are reported in Promtail logs, meaning it’s functioning as expected.

However, Promtail isn’t showing any “Pushing logs to client” messages, which means logs aren’t being pushed to Grafana Cloud Loki.

also, i checked connectvity as follow which seems ok:

mac@macs-MacBook-Pro auth % curl -u "{USERNAME}:{API KEY} https://logs-prod-028.grafana.net

OK%

Here are more details:

docker-compose.yaml:

services:
  auth-app:
    image: auth-app:latest
    # container_name: auth-app
    build: .
    ports:
      - "8080:8080"
    volumes:
      - ./logs:/app/logs  # Mount logs directory
    environment:
      - SPRING_DATASOURCE_URL=jdbc:mysql://mysql:3306/ral
      - SPRING_DATASOURCE_USERNAME={XXX}
      - SPRING_DATASOURCE_PASSWORD={XXX}
      - SPRING_PROFILES_ACTIVE=dev
      - JAVA_OPTS=-Dlogging.config=classpath:logback-spring.xml
    depends_on:
      - mysql

  mysql:
    image: mysql:8.0
    container_name: mysql
    ports:
      - "3306:3306"
    environment:
      - MYSQL_ROOT_PASSWORD={XXX}
      - MYSQL_DATABASE={XXX}
    volumes:
      - db_data:/var/lib/mysql

  promtail:
    image: grafana/promtail:2.9.0
    volumes:
      - ./logs:/app/logs
      - ./promtail-config.yaml:/etc/promtail/config.yml
    command: -config.file=/etc/promtail/config.yml

volumes:
  db_data:

promtail-config.yaml

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
  - url: https://logs-prod-028.grafana.net/loki/api/v1/push
    basic_auth:
      username: {USERNAME}
      password: {API_KEY}

scrape_configs:
- job_name: auth-app
  static_configs:
  - targets:
      - localhost
    labels:
      job: auth-app
      env: dev
      app: auth-app
      __path__: /app/logs/*.log