I noticed datasources errors in Grafana. When looking at the Grafana logs, I saw many errors like:
annotationruleID1dashactualerrorstrconv.ParseInt: parsing "": invalid ...level=error msg="Failed to save alert state" ... severity=critical" state="unsupported value type" error="pq: remaining connection slots are reserved for roles with privileges of the \"pg_use_reserved_connections\" role"
and
annotationruleID1dashactualerrorstrconv.ParseInt: parsing "": invalid syntaxlogger=datasources level=error msg="Failed getting data source" err="pq: sorry, too many clients already"
I logged into Postgress, and it turns out that there are only four connections from Grafana (two instances), with the maximum 262143 connections allowed.
I use Grafana 10.2.3 and PostgreSQL 16.3 on aarch64
I wonder if this a known issue or how it can be resolved.
1 Like
hi,
how many other connections do you have from other sources besides grafana?
the error message is indicating your db is out of connections, and only super users can connect, see below:
“You can reserve a small number of connections for the superuser
with superuser_reserved_connections, but there’s no way to do a
similar thing for any other user. As mentioned above, a CREATEROLE
user could set connection limits for every created role such that the
sum of those limits is less than max_connections by some margin, but
that restricts each of those roles individually, not all of them in
the aggregate. Maybe we could address this by inventing a new GUC
reserved_connections and a predefined role
pg_use_reserved_connections.”
out of the box, postgres typically is configured for 115 connections, 100 users and 15 superusers.
you should check what is using the other 96 connections, if there are only 4 grafana connections.
its also not recommended to just increase the number of connections, you also need to take buffer cache into account, and on linux changing some kernel params as well to cater for the additional memory requirements of more connections.
There are no other connections. The connection limit is 262143, and there are only Grafana connections. That’s why it’s very strange…
It turned out that the Postgres instance was too small to handle our Grafana instance.
2 Likes