What are normal grafana write io speed

I would like to ask what is good grafana write io speed?

Currently my prometheus is using around (99.04 K/s prometheus) and grafana 3.21 M/s grafana server.

is this normal?

Could someone share ways to debug

Using Debian 12 and grafana server version is Version 9.5.20

I also notice that when grafana goes to 3.21 M/s or more this message starts to appear in the log file
level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked"

I am using sudo iotop -oP to look for io speeds.

i removed all prometheus targets and leaved only one job from node_exporter to see is something is changed, i hoped when prometheus have low amount of data to collect and send the write speeds of grafana will go down because there is less information to write.

Unfortunately this did not helped :confused:

Grafana writing what? Other than its own log data not sure what else it is writing?


I am wondering that too…

Grafana debug logs, Grafana traces, strace, …

I guess you are using the default SQLite DB (which is a terrible choice for high traffic, but people are using it because “it just works” by default). Serious Grafana deployments use more serious DB (MySQL, PotgreSQL)