Extremely long response times from Grafana server

Grafana MSSQL
1

Every few queries takes around 10 seconds to respond.

Setup:
Grafana(v8.2.5) running on Windows 2k19 server (VM) 2CPU, 8GB Ram. Dedicated for Grafana only. No other applications running.

WebServer/SQLServer: Windows 2K12R2 Physical box, 8 core, 128GB Ram

The website has multiple pages with iframe embedded graphs that get their data from the SQL database listed above.

About 30% of the time the pages take around 10 seconds to load (as seen in the IIS logs on the web server). During these times a notice shows up on the bottom left of the browser window stating “waiting on ‘grafana server url’”. The rest of the time the response can be as little as 200ms.

Sometimes it’s the same page, sometimes it’s a different page. Seemingly random. This waiting message even shows up when navigating away from a page with grafana content to a page with no grafana content. There is never a delay when navigating between pages with no grafana content whatsoever.

We have DynaTrace installed on the Grafana server and the Grafana server cpu never exceeds 40% and memory never 60%. Grafana is only using 0.26% of CPU. Network traffic averages 70kbps on a 100Mbps connection. About 30 packets per second (send and receive).

Are their configurations on the grafana server that can improve and make the performance more consistent?

2

Welcome

Are you using stored procedures or straight raw queries? Also are you using any parameters and are the columns on which you are filtering on have indices?

What happens when you run these same queries outside of grafana straight in sql server itself via ssms?

3

I am running straight queries. The majority do not have indices.
An example:
SELECT (TimeStamp) as time, ((CurInletMax*1.8)+32) as MaxTemp FROM [EnvironmentalTrackingHistory] Order by [TimeStamp] ASC

When I run the queries in SSMS the response times are less than 1 second.

1 Like
4

now, please run the grafana visual outside of IIS/iFrame, as in bring up pure grafana by itself and see how long it takes to run the queries.

5

Ok, I discovered that this isn’t Grafana. Further investigation showed that the same delays occurred when going between pages that didn’t contain any Grafana content.

Thank you for your responses.

1 Like