Grafana 8.0 incompatibilities

Hold on, I don’t understand. Are you saying that it is not actually a panel one can use; it is only some kind of flag that prevents Grafana from upgrading? But if Grafana does’t upgrade, it means that it still can use the Singlestat panel, no? Can’t I just edit manually the JSON definition of the panel and change its type to singlestat after the upgrade? I remember that I was able to get the old Table panel with a hack like that…

It is indeed a proper standalone panel. In fact, it’s the same old Singlestat that has been extracted from the Grafana repository into its own repository.

When you start Grafana 8.0 for the first time, it will check to see if the Singlestat plugin is installed. If it is, Grafana won’t perform the migration to the new Stat panel model, and simply update the panel type in the dashboard JSON from singlestat to grafana-singlestat-panel.

You’re correct that you could edit the dashboard JSON manually retroactively, but I believe the panel models for the old Singlestat and the new Stat are incompatible. If you manually set the type after the migration, you’d be using the new panel model with the old Singlestat, which likely wouldn’t work.

That’s not a problem. “Wouldn’t work” just means that the various options wouldn’t translate automatically. But there aren’t that many of them; I can re-configure them manually. I just did that with one of the visualizations - changed the panel type to grafana-singlestat-panel and re-configured it (use total instead of the default average, show spark lines, full height, and what color) - and it works just like it used to. Thanks for your help; this problem is solved.

Now only the Worldmap panel problem remains. Hopefully, it will be fixed soon, because the boss has gone off the reservation, wants us to ditch Grafana completely and to make our own visualization in PHP… This is crazy - it will take months (not just the visualization, we’d have to re-design the backend database, then modify the honeypot to use the new database… It will be a hell of a lot of work and completely wasted IMO… Not to mention that the result will look ugly.

1 Like

I’m getting this too after our upgrade to version 8. The fix would be to put an ORDER BY in your query to whatever column is your “time” column in the SELECT. That should fix this error if your query is a timeseries query (and not a table query).

But I agree, the upgrade broke a lot of things for us including all pie charts.

No. Try it. It wont work. The problem is that the MySQL data source no longer returns Unix epoch times (which is what all plugins that use time series expect). Instead, it returns human-readable timestamp string (date, time) - which breaks everything that expects to see an integer there.

The pie charts are easily fixable, though. Just remove the part of the query that returns time and tell the panel to use table instead of time series. Sadly, this approach doesn’t work for the Worldmap panel.

OK, I have one more little problem.

One of my visualizations is a hourly bar chart. It still works but now it looks like this:

Look carefully at the legend. There is a word “value” there that’s not supposed to be present. The panel uses time series; I suspect that the problem will go away when the data source is fixed.

this is how we’d write your query (but we use postgres) this assuming that your timestamp column is some type of integer recording seconds from the epoch:

SELECT
timestamp as “time”,

– or
– date_trunc(‘day’, (TIMESTAMP ‘epoch’ + timestamp * INTERVAL ‘1 second’)) AS “time”,

country_iso_code AS metric,
COUNT(country_iso_code) AS value
FROM
connections,
geolocation
WHERE
$__unixEpochFilter(timestamp)
AND
connections.ip = geolocation.ip
GROUP BY 1, 2
Order by 1

jwollman, I’m afraid I don’t understand what you mean. Yes, the time is Unix epoch time, which means that it is long integer. And the problem is that instead of this, the MySQL data source is returning human-formatted time string.

Using timestamp AS "time", makes absolutely no difference and results in exactly the same error. Same with your second proposal, which I don’t even understand the purpose of the manipulation you’re doing there.

OK, Grafana 8.0.3 has fixed this particular issue. However, the bigger problem (the one that affects the Worldmap panel so badly) is still not fixed - I noticed that the fix has been moved as a milestone for version 8.0.4.

This has been kinda fixed in Grafana 8.0.2. You can’t set the color of the value to whatever color you want - but you can make it not colored. From the panel settings, select Stat stylesColor modeNone. (The options are None, Value, Background, with Value being the default.)

Same problem here. After upgrading to 8.0.3, almost all my time series panels that use MySQL as the datasource stopped working.

image

Adding “ORDER BY created_at” to the query makes the error disappear, but now I see a completely empty graph.

Query:

SELECT created_at AS "time", sky_gold_log_tags.value as 'metric', $aggregation_query as "id" FROM sky_gold_log
JOIN sky_gold_log_tags ON sky_gold_log_tags.log_id = sky_gold_log.id AND sky_gold_log_tags.key = 'product_game'
WHERE $__timeFilter(created_at) AND action = 'REMOVE' AND source = 'BILL' AND valid = 1
GROUP BY metric
ORDER BY created_at

Was anyone able to fix this?

Adding “ORDER BY created_at ASC” to all queries fixed the problem I described above.

2 Likes

First, you’re talking about a Graph panel. My problem is with the Worldmap panel - and doing ORDER BY there doesn’t help at all. One of the developers is working on updating the Worldmap panel to accept the new time format but it hasn’t been released yet.

Second, it is quite possible that your problem isn’t fixed, either, and it just so happens that the human-readable timestamps are ordered correctly by ORDER BY. Try playing with different time ranges, encompassing different months, etc.

The problem is in the MySQL data source of Grafana. It has to be fixed there. Some kludges might work for some panels some of the time, but they are not a real fix.

I can confirm that I have see ORDER BY <time_field> ASC fix a number of weird issues with MySQL queries in Grafana that return timeseries data, including in Grafana 7 with the Graph panel.

1 Like

FWIW, this was not fixed in Grafana 8.0.4. I noticed that the milestone for fixing it has now been moved to Grafana 8.0.5.

This is very unfortunate. The bug has been around for 3 weeks already, the fix was supposed to be in 8.0.3, then in 8.0.4 - and it’s still not fixed. I don’t know what the hold up is, but our visualization not working already made us miss an important report deadline.

I was hoping for this to be resolved quickly. I should have reverted to version 7.5 and blocked the updates of Grafana forever. :frowning: Almost every single past update has either broken something or made things more inconvenient by moving the user interface elements around - but these were minor annoyances that I was willing to live with. But totally breaking an important visualization and not fixing it for weeks is really pushing the limits of my tolerance…

They have closed my issue on GitHub, saying that version 8.0.6 fixes it. It does not. The Worldmap plugin still doesn’t work. Now they’re saying that I must write some kind of transform - which is not clear to me how to do - and, yes, I’ve read the goddamn docs on this subject.

Fuck this shit. Grafana is dead to me. I’ve reverted to version 7, frozen the package, and will never, ever update it again. This is the only way to make this problem go away and get your stuff working.

1 Like

Same problem for me on a graphe, even migrated in time series panel, still got "failed to convert long to wide series when converting from dataframe: long series must be sorted ascending by time to be converted " …

Also reverted to 7.5.10 …

@vbontchev the Geomap panel replaced the Worldmap panel in Grafana 8.1. All further development is focused there, I believe…

This worked for me on Grafana 8.1.1 and SQL Server 2019, but I think is not an engine related error. Greetings

This topic was automatically closed after 365 days. New replies are no longer allowed.