I’m looking into deploying grafana on an on-prem k8s cluster using the grafana helm chart.
I’d like to have it run in a HA configuration with several grafana pods replicas and I read that using a database is a must in such a case (Set up Grafana for high availability | Grafana documentation).
However, since the plan is to install grafana on k8s, is it possible to use a pvc instead of a database? If all the grafana pods will share the same pvc, wouldn’t that replace the need for a database? The storage class the pvc will use can handle multiple process access to files (sort of like an NFS).
Has anyone successfully deployed grafana in a HA setup on k8s with only a pvc?
we don’t want to conflate storage with application backend unless the application uses files in volumes to function (though sqlite is a file but you know what I mean)
We are excited that you joined our OSS community. Please read about some of the FAQs in the community
I talked to the Engineering folks and the short answer is that Grafana does need a Database.
Long answer:
Grafana needs a database. SQLite stores its DB on disk and thus can be stored in a PVC.
However, PVCs are per pod, so you can’t (generally) mount a PVC onto two instances. Then, there’s the fact that an SQLite database isn’t designed for concurrent writes, so even if you did, you’d get yourself into trouble.
I’ll look into what DB to use.
If I point Grafana to use for example a postgres StatefulSet cluster as its DB (which will have its own pvc for saving DB data over pod restarts), does this mean I can disable persistence in Grafana’s values.yaml?
Or is persistence still required if I want to save dashboards and alerts across Grafana pod restarts for a HA setup, assuming I enable persistence with pvc instead of using the sidecar+configMaps option?
If you deploy Grafana on K8s with the Helm Chart, you will define what plugins you need. Then each time it is started, it will install these plugins on the folder defined. So they don’t need a PV.
I have just migrated (two days ago) from SQLite on a PV to PostgreSQL on AWS RDS and removed PV. So far, everything is still working as it as before..
P.S. I am writing up a blog post about this migration and my experiences
P.S.2 Standard Disclaimer: Still I could not test all functionality in two days. There may be some issues that I will encounter in coming days, so I keep my options to change my feelings about PV
Please feel free to ask any further questions and issues
If you have configured Grafana to use DB, all alerting / dashboard data will be stored in the DB and you don’t need PVC for anything related for HA.
As I mentioned in the previous comment, I have just tested this scenario on my environment and everything so far works as before. No PVC, all data is in PostgreSQL.
Please feel free to ask any other questions or issues.
but you know what I mean)
community support forums !!
