We are monitoring the 200 servers using Status panel at grafana.
When we open the page, it is taking too much time to reload(15-30 minutes).
Server specifications which grafana contacts for metrics:
server type: Linux - HDD
The data at InfluxDB is 70GB.
While reloading the grafana, the RAM occupying by the InfluxDB is very high and occupying most of the cores.
We thought that, because of the high amount of data, it is failing to reload fast. So, we took the backup of data before 15 days and deleted the data at InfluxDB.
The disk space at InfluxDB reduces from 69 GB to 60GB only. But we have very high amount of data at that time.
Why the 9GB of data only reduced?.
Why this is happening?.
How to resolve the slowness issue?.
If you want any inputs, will update here.