Configure Fluent-bit to send logs to lokil

i don’t know if this is the right place but I need your help guys.

I’m using Loki 3.2.0 and fluent-bit 3.1. my goal is simple. I have 2 paths of the log to get monitored by fluent-bit and give them different tags and use those tags as a label to store in Loki. so I make this config

[INPUT]
    name tail
    path /var/log/gitlab/gitlab-rails/application_json.log*
    db   /etc/fluent-bit/application.db
    tag  application
    exclude_path /var/log/gitlab/gitlab-rails/application_json.log*.gz

# Read interval (sec) Default: 1
    #interval_sec 1

[INPUT]
    name tail
    path /var/log/gitlab/gitlab-rails/api_json.log*
    db   /etc/fluent-bit/api.db
    tag  api
    exclude_path /var/log/gitlab/gitlab-rails/api_json.log*.gz
    #interval_sec 1

[FILTER]
    name  modify
    match application
    add   job application

[FILTER]
    name  modify
    match api
    add   job api

[OUTPUT]
    name   loki
    match  *
    host   192.168.11.20
    uri    /loki/api/v1/push
    port   3100
    labels agent=fluent-bit, job=$job

but something strange happens. when I tried to pull the data from the last 24 hours from my Grafana dashboard, the logs shown in the panel weren’t the same as the log file. for example: the oldest log line in the file was 2024-11-05T04:00:05.882Z and from my dashboard, the oldest log line is 2024-11-05T12:45:07.040Z

in short, I assume that fluent-bit didn’t send all the logs in the file to Loki. what possibly can cause this? or am I doing something wrong?

Thanks

while I compare it to promtail, it’s better. the latest and the oldest log lines, all have been sent to Loki

this is the latest logline from the file
image

this is the oldest logline from the file
image

If you are seeing logs with promtail but not fluentbit, then problem is most likely not Loki.

Do you expect your files to be dropped in? Perhaps not use tail in your file input?