Grafana Cloud Prometheus showing data 5-minutes in the future

Hi, we’re forwarding database stats into our Grafana Cloud Prometheus server for monitoring, and I’ve noticed that we are seeing about 4 minutes of future data in graphs (e.g. for cpu_usage_sustem{})

The data appears to just be a flat continuation of the most recent data point… each point updates with the actual data as the minute rolls around, but its a bit bizarre…

Is there some setting we can change for this or do we just have to make our queries always run from the current time?

Hey there @howardcox if you’re seeing future data or data with incorrect timestamps, I think that may be an issue with the system time on the server where Prometheus/Grafana Agent/etc. is collecting data from.

Can you investigate if this is the case?