Entry for stream too new

Grafana Loki
,
1

Bonjour :waving_hand:

`grafana/alloy:v1.12.1 (docker)`

`grafana/loki:3.6.3 (docker)`

Fail2ban install on VPS.

I have this error log always, always, every times:

{
    "ts":"2026-01-05T10:25:30.375026104Z",
    "level":"error",
    "msg":"final error sending batch,
    no retries left,
    dropping data",
    "component_path":"/",
    "component_id":"loki.write.grafana_loki",
    "component":"client",
    "host":"loki:3100",
    "status":"400",
    "tenant":"",
    "error":"server returned HTTP status 400 Bad Request (400): entry for stream '{component=\\\"actions\\\",
    exporter=\\\"OTLP\\\",
    geoip_city_name=\\\"Ashgabat\\\",
    geoip_continent_code=\\\"AS\\\",
    geoip_continent_name=\\\"Asia\\\",
    geoip_country_code=\\\"TM\\\",
    geoip_country_name=\\\"Turkmenistan\\\",
    geoip_location_latitude=\\\"37.95\\\",
    geoip_location_longitude=\\\"58.3833\\\",
    geoip_subdivision_code=\\\"S\\\",
    geoip_subdivision_name=\\\"Ashgabat\\\",
    geoip_timezone=\\\"Asia/Ashgabat\\\",
    instance=\\\"domaine.tld\\\",
    job=\\\"fail2ban\\\",
    priority=\\\"NOTICE\\\",
    service_name=\\\"actions\\\"}' has timestamp too new: 2026-01-05T11:25:22Z"
}

has timestamp too new?

I see in logs:

“ts”:“2026-01-05T10:25:30.375026104Z”

and

2026-01-05T11:25:22Z

OK, one hour time difference but how to change?

My host is on timezone Europe/Paris, and docker is on UTC date.

I found in Grafana Loki configuration parameters | Grafana Loki documentation

`creation_grace_period`

But is it the recommanded way to avoid this time problem?

Thanks

2

That indicates that something has wrong UTC time. Check UTC times (not local times) on boths servers - they must be the same. Or someone is sending local time without timezone and than Loki may assume that UTC timezone actually.

3

OK thanks.

On receiver:

timedatectl status
               Local time: Tue 2026-01-06 14:27:50 CET
           Universal time: Tue 2026-01-06 13:27:50 UTC
                 RTC time: Tue 2026-01-06 13:27:51
                Time zone: Europe/Paris (CET, +0100)
System clock synchronized: yes
              NTP service: active
          RTC in local TZ: yes

On sender:

 timedatectl status
               Local time: Tue 2026-01-06 14:27:29 CET
           Universal time: Tue 2026-01-06 13:27:29 UTC
                 RTC time: Tue 2026-01-06 13:27:29
                Time zone: Europe/Paris (CET, +0100)
System clock synchronized: yes
              NTP service: active
          RTC in local TZ: yes

Somes seconds of difference is delay betwwen run command on differents terminals.

Only in alloy docker, timezone is different.

I increase create_grace_period to 3h to solve problem. But I don’t know if it’s recommanded solution.

4

IMHO you sorted this problem, but you may create new problem: e.g. time of log line in the Loki is one hour shifted before/after actuall time when log line was generated

1 Like