Getting "entry too far behind" even if reject_old_samples and reject_old_samples_max_age should allow

When you already have logs in a log stream reject_old_samples_max_age is irrelevant.

When you try to write older logs into a log stream (log stream defined as logs with the same set of labels) that already has newer data then it’s considered out-of-oder writes. Loki has a limit of (max_chunk_age/2) when it comes to out-of-order write, and with default 2h of max chunk age that comes down to 1 hour. Essentially, if you have logs in a log stream, the oldest data you can write to the same stream is something 1 hours older than the latest log.

See Grafana Loki configuration parameters | Grafana Loki documentation