Random "data outside time range" after upgrade

I recently upgraded to the latest version 6 from version 5, not sure anymore which subversion of that. However I am running into a really strange problem. Sometimes graphs for which I know there is new data, won’t update. Then when I change the time picker, it does, even though for example initially the window is for example 2 days, and I get “data outside time range”, and then I go down to 24h, and all of a sudden there is data in the graph! The same effect can be seen when just changing the “To” field to a different offset, for example changing it from “now-10m” to “now-9m” will make the graph show data again.

Here you can see that there’s this error on the 24h window:

But when I got down to the past 12h all of a sudden data can be found:

It’s driving me absolutely nuts as I don’t see any errors anywhere, it’s more or less the same on all browsers. But it is totally random it seems. Sometimes it happens, somtimes it doesn’t, and on different graphs using similar data sources it’s also mixed results. I’m using Influx 1.7.9 on Ubuntu 16.

Any help greatly appreciated!

You say it is random, do you mean that on the example you gave that if you switch backwards and forwards between the 24 and 12 hour ranges that sometimes it shows on the 24 hour range and sometimes it doesn’t or as you switch backwards and forwards it never shows on 24 hours, but if you do the same at another time it will always show on the 24 hours range?

Actually no, in the example above, switching between the two ranges keeps giving the same result. But I just found out that I am getting the same problem on a completely different install on another server. What I mean by random is that for example 2 graphs that come out of the same data source, side by side on the same dashboard, would sometimes have one of the graphs show okay, and the other would be cut off at some time in the past.

I’ve just made some progress, I believe. I have cleared all browser data from one browser I was using (Safari) and that has made the problem go away for now. Is there a toggle to make the browser be less smart about local caching here?

If clearing the cache changes what is displayed then I don’t understand why. Someone else will have to help.

it’s still happening, in every browser. Clearing the cache works once, and then after a while it is the same. For example:

It’s very consistent on this graph. Setting the window to 2d shows “data outside time range” and then setting it to 24h shows the data as there is in the influxdb.

And setting it to 7d shows there is a cut off on the 7th, and it’s not showing data that is showing up in the 24h window.

Even weirder is when I change just the time offset from -2h to -1h, the 2d window then does have data showing:

How can this be? How can I possibly debug this? This never happened before the upgrade to version 6. Is nobody else having this problem?

Can you show us the query you are running please? Also what does that query show in the Query Inspector for a fault situation?

The query for example is:

“SELECT sum(“qps”) FROM “1_year”.“tld_hour” WHERE (“org” = ‘’) AND time >= now() - 2d and time <= now() - 5m GROUP BY time(1h), “node””

The query inspector for this one is quite large and contains a lot of sensitive data, but it has loads of arrays like these:

I just upgraded to 7.0.5 and the problem is the same. When I look at the query inspector after a roll over into a new 5m period (I’m grouping per 5m) and run a refresh I don’t see the new data show up in the result objects. However when I run the query by hand on the InfluxDB CLI I get the right results.

And as usual, when I change the time picker to a slightly different time frame, it does retrieve the latest data.

As you can see the influx CLI yields data past ‘139035’ while it’s not showing anything past that value in Grafana, no matter how often I press “refresh”. Only changing the time picker to another window size will somehow force a true reload for fresh data from influxdb.

I also just turned on query logging in influxdb, and I indeed see no queries from Grafana when the graph isn’t being updated, but I do see them when I change the time picker.

How can this be? Nobody else with this problem?

I just found that if I set the “Time Shift” field in the query options in Grafana to “0h”, that Grafana does query every time upon a reload. Like so:

image

this smells like a bug to me. Why would it work when it’s set to “0h” and not when it defaults to “1h”? Why does it default to that anyway?

As a global workaround, is it possible to set this “Time shift” option for the datasource by default? I only see this option for the data source for each graph individually.

anyone have a clue on this one? why would the time shift option “fix” this?