X509 everywhere when using Azure Mysql

Hi, we just deployed a Grafana 8.4 as a container on Azure, and everything is working fine. Then we create a Mysql Managed database to store the Grafana configuration.

We use the configuration:

GF_DATABASE_SSL_MODE: true
GF_DATABASE_CA_CERT_PATH: /etc/ssl/certs/DigiCertGlobalRootCA.crt.pem
GF_DATABASE_TYPE: MySQL

And we’re able to connect to MySQL and store our data.

The weird thing is that all the other data sources show a x509 certificate signed by unknown authority.

For example, the managed Elasticsearch running on Azure, Azure monitor…

All our connections to an SSL service are showing a x509 error.

Do you know what we should do to correct this behavior?

Thanks

Datasources have own TLS management usually:

You may try to add CA certs to your OS CA certs, so they then can be recognized by Grafana binary.

Hi, in fact we use mysql as the internal storage of Grafana not as a datasource.

When using Mysql as an internal database, we can’t connect to other SSL/TLS services. For example we need to skip verify Elasticsearch datasource and we can’t connect to Azure Monitor:

As a side note, we don’t have this issue when we deploy a Grafana with self database (SQL LITE) so I’m quite sure it’s related to the usage of Azure MySQL but I’ve no clue of what’s happening or what we may change on Azure MySQL configuration…

You didn’t show your full config. But I guess you are overwriting whole /etc/ssl/certs/ folder in the container, so container doesn’t recognise other CAs then :man_shrugging:

I don’t understand why you need DigiCertGlobalRootCA.crt.pem, when Digicert is standard CA and standard Grafana images already have all standard CA certs, e.g.:

root@play:~# docker run --rm --entrypoint sh -ti grafana/grafana:8.4.0
/usr/share/grafana $ ls -lah /etc/ssl/certs | grep -i 'DigiCert_Global_Root_CA'
lrwxrwxrwx    1 root     root          35 Feb 16  2022 3513523f.0 -> ca-cert-DigiCert_Global_Root_CA.pem
lrwxrwxrwx    1 root     root          62 Feb 16  2022 ca-cert-DigiCert_Global_Root_CA.pem -> /usr/share/ca-certificates/mozilla/DigiCert_Global_Root_CA.crt
/usr/share/grafana $ openssl x509 -text -noout -in /usr/share/ca-certificates/mozilla/DigiCert_Global_Root_CA.crt
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            08:3b:e0:56:90:42:46:b1:a1:75:6a:c9:59:91:c7:4a
        Signature Algorithm: sha1WithRSAEncryption
        Issuer: C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
        Validity
            Not Before: Nov 10 00:00:00 2006 GMT
            Not After : Nov 10 00:00:00 2031 GMT
        Subject: C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA

Yes you’re right, just found that the whole folder was bind mounted instead of just the Mysql certificate. So the certificates folder is empty, except for one Mysql certificate. Thank you for your help

1 Like

It will be nice if you close your issue (Product Area: x509 errors on all datasources · Issue #80994 · grafana/grafana · GitHub )
Anyway, it was not a Grafana issue, but your configuration problem.