We are a company that is new to using Grafana and we have come across an issue with our dashboard that we would like to discuss with the community.
Currently, we are using the online version of Grafana (not the installed version). Our situation is as follows: we have a large query with multiple joins running in one panel. The other panels (~20 approximately) rely on this first panel to retrieve their data. Sometimes, when we enter the dashboard, the initial panels load correctly. However, when we scroll down to view the remaining panels, all the panels display the “loading bars” and never finish loading. It seems that scrolling down triggers a reload of all panels, causing them to get stuck in a loading state.
One potential solution we have considered is optimizing the query, although we believe there might not be much room for optimization.
We have also noticed that upgrading to the cloud pro version (currently using the free version) would provide us with “Query caching” and other features. The example we found here regarding query caching mentions multiple users running the same query. However, would it also help in our situation?
We wonder if any of the other “PRO” features could help resolve our issue, or if we would encounter the same problem even with the PRO version.
We would greatly appreciate any assistance or suggestions to help us resolve this issue.
Thank you for your time.
Welcome to the community support forums !!
We are excited that you joined our OSS community. Please read about some of the FAQs in the community
I believe that you are using the Grafana Cloud (free version) which you mention as “the online version of Grafana”
What is your Data Soruce and which panel visualization you are using?
Hi @usman.ahmad ,
Thanks for your reply!
You’re right, we’re using the Grafana Cloud free version. Besides, our data source is MySQL, and the panel visualization that is loading the original query is of type “Stat”. Then, we have other panels (arond 20) of type “Stat”, “Gauge”, “Gauge bar” and “Table”. These other panel visualizations are conected to the first panel as their database.
Thanks for the feedback.
So, in terms of optimization here are some information.
Yes, optimizing the query could help as Grafana will display the visual graph from Data Source and it depends how much the complex query you have. But as you said that you already did some optimisation on the query level.
What you can also do is perform optimization on MySQL server configuration side e.g. increase timeouts and sessions. I do know a good 3rd party tool called MySQL Tuner script which simply provide suggestions for possible improvements.
The second thing you can also do is reduce the amount of Transformations on the Grafana UI side. For e.g. some users use sort transformations to sort the data in ascending or descending order which can be done also on the query level.
The lesser the transformation the lesser network bandwidth use.
One more tip is to use rows. i.e. create rows for a similar set of panels e.g.
Row 1 is only for memory related and Row 2 is for CPU related. What it will do is that as save you rows in collapsed (not expanded) after moving the panels into it and a user want to view the dashboard, can expand the row as per requirement and since if you have e.g. total 5 rows and only 2 are expanded then the remaining 3 will not get loaded and will again increase the chance. [link to a blog about Rows]
The query cache option is definitely helpful and useful once the above options still causes issues in viewing the graphs.
I would suggest to ask our team using this link so that they can help you
Hi @usman.ahmad ,
Thank you very much for your time and patience to explain the different options here.
We’re probably trying to work on optimizing the query (if there’s room for improvement at all). However, we’re not encountering any timeout errors, therefore the MySQL server doesn’t seem to be an issue here.
Regarding the transformations, we are only using one panel with the query to retrieve the data, therefore on the rest of the panels, we point to that first panel, and apply transformations to group and sort the data. Otherwise, we would need to create multiple queries, and we would be therefore increasing the problem.
We’re not sure about the rows thing. We’ve read the link you kindly provided, but it seems to be a valid option when the metrics are repeating through panels. Thats not our case, I think, but we’re not 100% sure about the functioning of that feature.
Regarding the query cache, we’re going to use the link you provided to contact the team, so they can tell us if, besides helping when multiple users are using the same query, does it help in our situation? (just one user launching the query, but it’s a very heavy query).
Again, thank you very much for your time with us!
Have a nice weekend.
Yes, please ask in the link if the team can give you some more information as I do understand that the question is about performance, and in Grafana Enterprise there are some features that can help but will also need to get some feedback from an Engineer perspective to get a better idea.
From Grafana OSS side this is all I can help
Have a nice weekend !!