i don’t know if this is the right place but I need your help guys.
I’m using Loki 3.2.0 and fluent-bit 3.1. my goal is simple. I have 2 paths of the log to get monitored by fluent-bit and give them different tags and use those tags as a label to store in Loki. so I make this config
[INPUT]
name tail
path /var/log/gitlab/gitlab-rails/application_json.log*
db /etc/fluent-bit/application.db
tag application
exclude_path /var/log/gitlab/gitlab-rails/application_json.log*.gz
# Read interval (sec) Default: 1
#interval_sec 1
[INPUT]
name tail
path /var/log/gitlab/gitlab-rails/api_json.log*
db /etc/fluent-bit/api.db
tag api
exclude_path /var/log/gitlab/gitlab-rails/api_json.log*.gz
#interval_sec 1
[FILTER]
name modify
match application
add job application
[FILTER]
name modify
match api
add job api
[OUTPUT]
name loki
match *
host 192.168.11.20
uri /loki/api/v1/push
port 3100
labels agent=fluent-bit, job=$job
but something strange happens. when I tried to pull the data from the last 24 hours from my Grafana dashboard, the logs shown in the panel weren’t the same as the log file. for example: the oldest log line in the file was 2024-11-05T04:00:05.882Z and from my dashboard, the oldest log line is 2024-11-05T12:45:07.040Z
in short, I assume that fluent-bit didn’t send all the logs in the file to Loki. what possibly can cause this? or am I doing something wrong?
Thanks