Yes, I have seen all the tips on how to avoid it. The thing is this annoying thing keeps popping up randomly. If I refresh the browser, changes are good that it goes away. If I click on ‘Zoom to data’ it zooms in to exactly the current time range.
I just want to provide a dashboard for the end users that shows the data without this annoying overlay. The data it shows is correct, but the dashboard is unusable just because of this ^%&$^ overlay.
You show way more critical errors in a small, subtle icon on the top left of a panel. Why on earth do you hide critical errors, but this annoying warning needs to block the entire chart? I really don’t get it. And don’t get me wrong. This message has value. Just not that value that it blocks the otherwise perfectly usable chart.
An alternative: Just provide an option on the panel to get rid of this warning. I get it that something is wrong with the data source or Grafana just believes this for some reason. OK, but I want to ignore it!!!
Or at least give this overlay a unique ID, so that I can hide it through CSS. But even that possibility is blocked:
After reloading the page this overlay is gone (problem fixes itself?) while showing the exact same data or just clicking on edit as you can see below, where the overlay is also gone.:
You can see in the Victoria Metrics queries, that I am even explicitly filtering for the time, which should not be necessary as for Victoria Metrics and Prometheus datasources Grafana automatically passes start and end time parameters to the API already. But even that does not help to solve these regular hick-ups.
You can see that Grafana passes the time range to Victoria metrics in UTC on the top right.
This problems appears particularly when using ‘Yesterday’ or ‘Day before yesterday’, so time ranges with full 24 hours of data. I suspect that the last data point may sometimes cause this problem. But then it’s a Grafana bug that must be solved, if it cannot deal correctly with 24hour data.
But anyway. Even if not filtered correctly, I want to get rid of this stupid warning overlay and just show the data. Sorry, but I am fighting this stupid overlay since years in so many occasions, where filtering never solves this reliably, but Grafana is so stubborn to insist on showing it, even though I know that everything is fine.
It sounds like a problem how Victoria Metrics return timeseries (debug with query inspector - if there some timestamp in the query reponse, which is outside of requested end/start timestamp).
AI recommendation (maybe dumb som of them):
Querying fixed 24-hour intervals like “Yesterday” or “Day before yesterday”, the last data point often causes VictoriaMetrics to return a trailing time series or data point outside your requested range due to how inclusive range matching, step alignment, and lookbehind evaluation interact at the query boundary. [1]
Why the Last Data Point Triggers This Issue
Inclusive Upper Bounds (end parameter):
By default, query_range executes evaluations at exact intervals: start, start + step, start + 2*step, up to start + N*step. The end timestamp is strictly inclusive if it lands perfectly on an evaluation step. If Grafana or your script requests an explicit end timestamp (e.g., exactly 23:59:59), and a raw database sample matches that last millisecond or step exactly, VictoriaMetrics will evaluate and return that final edge data point. [1, 2]
Lookbehind Interpolation Maxing Out:
If your PromQL query uses sub-queries or range vectors (e.g., rate(metric[5m])), VictoriaMetrics must evaluate the data window trailing behind that final upper-bound timestamp. If it finds any data point right on the boundary, it computes it, which can make it look like the engine is overreaching past the intended stop time. [1, 2]
Response Cache Timestamp Snapping:
VictoriaMetrics optimizes performance by caching query_range requests. To make cache hits possible, it rounds and aligns the start and end arguments to the nearest clean step interval. If you select a 24-hour block, the engine may snap the final timestamp slightly forward to align with the step, causing it to grab a raw data point that technically belongs to the first few seconds of “today”. [1, 2, 3]
How to Fix It
Depending on how you are querying VictoriaMetrics, you can apply these target fixes to restrict that trailing edge data point:
1. Shift the Grafana Time Window (If using Grafana)
If you are using the native Prometheus/VictoriaMetrics datasource, the built-in time macros can include the very last millisecond of the next day. You can use math inside your panel’s Relative time or within your custom query to back away from the edge:
Set Relative Time: Instead of now-1d/d, use a custom shift like now-1d/d to now-1d/d+23h59m to prevent the query from catching the midnight boundary transition.
2. Disable Cache-Snapping for Boundary Queries
Force VictoriaMetrics to evaluate the absolute exact millisecond you requested instead of rounding the upper boundary to a clean cache step. [1]
If VictoriaMetrics is over-interpolating trailing metrics at the end of the day because it is searching too far into the gaps, you can tighten how far it searches for raw data points. [1]
Reduce the lookbehind window by adjusting the global configuration flag on your VictoriaMetrics single-node or vmselect component:
bash
-search.maxLookside=5m
If it is annoying just for you, then I would try to write browser script (e.g. Tampermonkey)/extension to remove only that div `data-testid=“time-series-zoom-to-data”.
You’ve already ruled out the usual time-filter/timezone problem. The important clue is that the data is correct, the overlay appears intermittently, and refreshing the dashboard removes it without changing the data.
Your DOM inspection also shows a practical workaround. The overlay contains:
button[data-testid="time-series-zoom-to-data"]
so you can target the overlay through that button instead of relying on Grafana’s generated CSS classes:
div:has(> div > button[data-testid="time-series-zoom-to-data"]) {
display: none !important;
}
Based on the DOM in your screenshot, this should hide the “Data outside time range / Zoom to data” overlay while leaving the graph and its data visible.
The remaining issue is how to inject that CSS into Grafana. Grafana 13.1.1 does not provide a general custom-CSS setting in grafana.ini; content_security_policy_template configures CSP and isn’t a CSS injection mechanism.
So if you control the users’ browsers, a user stylesheet/browser extension such as Stylus is a straightforward way to apply the rule.
This is a UI workaround rather than a fix for why Grafana is triggering the warning, but for your stated goal showing the correct dashboard data without the blocking overlay it should hide the overlay while keeping the graph visible.
So very awkward workarounds with several drawbacks (user’s cannot simply use ‘Yesterday’ from the time picker, system needs to disable caching) just because Grafana is not handling Prometheus and Victoria Metrics data properly or because a warning is displayed in a way that makes the chart unusable.
My proposal: Grafana should fix this behavior.
Simple fix: Move this warning into the subtle error icon overlay.
Better fix: Allow to ignore this warning
Best fix: As Grafana is doing the timefiltering be less strict, before showing this warning, but give the data a grace period before and after the time range to handle a Prometheus/VM feature properly.
bingo. Can recommended one off fixes really address the root cause issue and can it be applied to 100+ devices across a big organization? Like you said best to file a bug.
Workarounds will eventually break down the line if something underlying changes like a newer version of grafana.
Not sure you know how Prometheus queries work? Typically in the easiest form you just type the metric name. Grafana passes the time frame to the API. That’s it. You don’t add filters for the time frames on top of it.
But even such a minimalistic, easy and straightforward setup is causing such intermittent issues again and again, just because the way how Prometheus and Victoria Metrics are handling the values that hit the boundaries.
Query options are the defaults suggested by Grafana. You can see that the selected metrics are very plain and simple. As said earlier: With Prometheus/Victoria Metrics Grafana is sending the time frame to the API to filter. So in my view Grafana is unable to even handle very simple, straightforward examples properly.
Did you file a bug with grafana? Or if a paid customer you should call them
Another option would be checkout the code for victoria metrics data source, to see where the issue might be happening and suggest a change or better yet contribute code for the change you recommend.
That is a clean reproduction → plain query, no manual filters, only the preset changes. This rules out anything on the query/datasource side, it’s Grafana’s own boundary logic misbehaving for that specific relative range.
Worth filing as a GitHub issue against grafana/grafana core
I see in your original full dashboard there appears to be whats looks like a zoomed in panel section right at the bottom. This appears to possibly involve mixed time ranges. Maybe you should disable that specific panel and try again as Grafana might be detecting to the lower zoomed time range. A shot in the dark but then things like these are sometime so obvious we tend to overlook?
All queries apart from the last use the same time range. I have had problems like this when the actual time series data is a mismatch to the selected option.
Otherwise best to use a panel with mutliple charts and using relative addressing so you can see full data plus day before etc in one dropped down panel.
