Grafana / Server Error - Failed to get settings

Dear all,

starting from a couple of days ago we’re having some problems getting inside Grafana Cloud.

Intermittently, trying to reach our favorite dashboard, we are experiencing the following problem:

Grafana / Server Error
Failed to get settings
Check the Grafana server logs for the detailed error message.

Please checkout the screenshot:

We use Elasticsearch as datasource, is there any way to check logs? Or could you please help us with this?

Did this issue happen to anyone else here?

Many thanks in advance,
Mirko

Same error since begining of december aprox. We use Amazon AWS as Datasource.

Carlos

From the beginning of this week, we’ve been seeing a mix of these error messages with serious frequency.
-Grafana/Server Error Failed to get settings
-Unauthorized
-Unable to load metadata

Hello guys,

many thanks for your quick response, because it excludes any configuration issue I might have introduced and I was starting to think/struggle with.

Let’s wait for a fix and/or further details from the development team.

Thank you for bringing it to the surface, as we had been wrestling the past few days with it and figured that if it wasn’t reporting in the forums than it must have been our issue!

Before we go waste more time with it, I figured I’d ask…have you tried any ‘workarounds’…re-adding your data source, recreating the dashboard, etc.?

Thanks for reaching out, we are investigating problems with some Hosted Grafana instances being unable to reach their databases, and will update this thread as we track down and resolve the cause of the problem.

1 Like

No, I didn’t try any workaround yet but, given the last post from @dcech I would rather wait for their response.

@mmw et al, we have made some adjustments to the database service, and are not seeing any more errors in the logs. Thanks for your patience, please let us know if you have any more problems with your Hosted Grafana.

Hi @dcech,

thank you very much! :muscle:

I’m gonna keep checking it time by time!

Cheers,
M.

Hi @dcech. I started seeing the same error two days ago, without any configuration change. I’ve noticed it for the first time in 6.0.2 but now also in 6.1.4 on my self-hosted Grafana server. The dashboard error comes together with log entries:
t=2019-04-18T10:08:26+0200 lvl=eror msg=“failed to look up user based on cookie” logger=context error=“database is locked”
t=2019-04-18T10:08:26+0200 lvl=eror msg=“Failed to get settings” logger=context userId=0 orgId=0 uname= error=“database is locked”
t=2019-04-18T10:08:26+0200 lvl=eror msg=“Failed to get settings” logger=context userId=0 orgId=0 uname= error=“database is locked”
t=2019-04-18T10:08:27+0200 lvl=eror msg=“Could not load alerts” logger=alerting.ruleReader error=“database is locked”
t=2019-04-18T10:08:27+0200 lvl=eror msg=“Failed to get settings” logger=context userId=0 orgId=0 uname= error=“database is locked”
t=2019-04-18T10:08:27+0200 lvl=eror msg=“Failed to get settings” logger=context userId=0 orgId=0 uname= error=“database is locked”
t=2019-04-18T10:29:27+0200 lvl=eror msg=“failed to look up user based on cookie” logger=context error=“database is locked”
t=2019-04-18T10:29:31+0200 lvl=eror msg=“Failed to get system stats” logger=metrics error=“database is locked”
t=2019-04-18T10:29:32+0200 lvl=eror msg=“Could not load alerts” logger=alerting.ruleReader error=“database is locked”

Can you share an insight what you adjusted in the database service?

The “database is locked” error indicates a problem with your sqlite database. This can happen if the database is left in an inconsistent state after a crash, or if there are problems with the disk.

One thing you can try is to dump the data from your existing db file into a new one, then swap it out. From inside your Grafana data directory (after shutting down Grafana):

sqlite3 grafana.db '.clone grafana-new.db'
mv grafana.db grafana-old.db
mv grafana-new.db grafana.db
1 Like

Thank you very much for the suggestion. So far so good. I will confirm the stability in a few hours.Thanks again.

I confirm that the proposed solution works. Thank you for your assistance!

1 Like