Want Zero downtime on grafana

Grafana Configuration
, ,
1

hi,
we are running grafana on AWS ECS service which has deployment configuration “Minimum healthy percent 100” & " Maximum percent 200" so that when we deployed any this on it 1st task will comes up then kill older task in such a way i want to achieve zero downtime for it. But when i tested it out it will goes down for 30 seconds (for 30 seconds i am not able to login to grafana). In docker logs i get this error -
logger=context t=2023-05-04T13:50:35.17526742Z level=warn msg=“failed to look up session from cookie” error=“user token not found”
logger=sqlstore.transactions t=2023-05-04T13:49:55.817085423Z level=info msg=“Database locked, sleeping then retrying” error=“database is locked” retry=0 code=“database is locked”

Can someone help me here? how can i achieve zerodowntime

THANK YOU!!

2

That sounds like you are using shared filesystem (and SQLite DB there for Grafana DB). Serious deployment should use serious database, which is able to handle concurrent access. You are on AWS, so RDS (PostgreSQL, MySQL) should be the best native solution.

3

Hi @swapnilkanere

In addition to what @jangaraj said, we do have documentation on how to set up high availability for Grafana.

Also, stated in the documentation that:

the default embedded SQLite database will not work, you will have to switch to MySQL or Postgres.

I hope this helps you further.