I’ve identified a behaviour in the dashboards I’ve created I don’t know if it’s the correct one.
I’ve made a dashboard in Grafana obtaining the result from Loki logs and when I select the filter time in the bar from Grafana, it doesn’t make the graph correctly.
It seems as if doesn’t “paint” the last sampling.
For example; If I select in the time filter 30 minutes, the graph is the following.
Also double check and make sure there are actually logs going into Loki with timestamp during those 7 minutes. We have some log pipelines with heavy caching and processing that delays for a minute or two. But 7 minutes is probably some sort of configuration issue on Loki, but still good to double check.
From your screenshot it looks like you are highlighting the timestamp “string” present in your logs, not the actual timestamp. If you expand on the log message it’ll tell show you the real timestamp, what do those look like?
I still don’t think that’s your log’s timestamp in Loki.
In version 9 Grafana used to show you timestamp in nanoseconds as a field when you expand the log. I don’t see that anymore. But from our own logs it looks like the first part of message is the timestamp (at least for us).
Also, have you tried to perhaps visualize it in a timeseries graph and see if it exhibits the same behavior?
You can query against Loki with an API, which will return actual timestamps for each log. Alternatively you can also look at query inspector in Grafana, which should have the API call and the actual text return.
Thanks a lot for your explanations, you’re right.
The timestamp is according to the data Grafana shows. I made the test at first and the data in the log and the time were the same, that’s why I’ve hidden the time in the Grafana log and I confused the timestamp with the logline as you said.