How are secret_key, data_keys table and secrets table involved in encrypting data source password?

Grafana Developers & API
1

Hi,

I have a Grafana setup using a PostgreSQL database (grafana-db) to store the Grafana configuration (dashboards etc.) and another PostgreSQL database (the to be data source) for storing data to be displayed (monitoring-db).

I would like to create a script to automatically provision the grafana-db with users, organizations, but also data sources. For this, I understand, that I need to encrypt the data source password and provide it the (correct) encrypted form. What I understand from Configure database encryption | Grafana documentation, there is the “main key” (secret_key setting in the configuration file), data keys, and the actual, encrypted, password (data source password, stored in data_source.secure_json_data?). I searched the Grafana source code, was able to find the information on how to extract the key_id from a base64 encoded “secret” - but what exactly is the secrets table now? How does it relate to secure_json_data, how does it relate to the other stuff?

And, decrypting the payload of both, secure_json_data and secrets entry using the decrypted data_key does not reveal anything readable.

In the source code, it looks like that secure_json_data is actually ignored for decrypting the values, it looks like it is taken from the secrets table, or am I wrong here?

It would be great if someone could describe the way how encryption/decryption is done in a more detailed way as it is currently in the documentation. Most important would be the clarification of the actual roles of the elements mentioned above. Why do we have two sources of secrets, data_source.secure_json_data and secrets.value? The values start identically, but have different length etc. Both seem to refer to the same key in data_keys, but what’s with the remaining stuff which I coulnd decrypt yet?

Thanks for you help!

Best regards
Matthias

  • What Grafana version and what operating system are you using?

11.4.0 in Docker (on Ubuntu 24.10, but doesn’t matter I guess)

  • What are you trying to achieve?

I want to provision a Grafana instances’ users, orgs and data sources purely using the database.

  • How are you trying to achieve it?

I’m currently at the level of being confused about the different sources of encrypted data and their relation to each other.

2

Could you explain it, pls?

3

I basically mean issuing SQL INSERTs and UPDATEs from an external tool to create entries in the user, org_user, org, and data_source (and any other relevant) grafana-db PostgreSQL tables.

4

That’s very fragile. You will need to spend a lot of time with reverse engineering (because this is not supported officially) and then your SQLs can be broken, when future Grafana release will update DB structure.

1 Like
5

I’m aware of this risk. Anyway, I still would like to understand it.

6

Generally that secret_key should be used for symetric encryption of secret values in the DB. That’s basic default encryption.
Of course admin can use more advance envelope encryption and then it will be more complicated.

You need to find what kind of encryption you have. Then you need a reverse engineering to understand DB relations/fields. I would use UI (e. g create datasource) and DB check after each operation what/where was changed in the DB.

7

There are a lot of business rules baked into the code/middle tier you would be bypassing by doing straight CRUD via scripts. Instead use the official grafana rest API which lays on top of the middle tier otherwise you will guaranteed have a wounded grafana instance

8

You need to find what kind of encryption you have. Then you need a reverse engineering to understand DB relations/fields. I would use UI (e. g create datasource) and DB check after each operation what/where was changed in the DB.

That’s exactly what I am in the process of. But this is poorly documented, therefore I’m not really getting the point how it is related. E.g., it seems to be redundant for me to have the secrets table and the password entry in data_source.secure_json_data. Therefore I’m search additional info here.

9

I’m absolutely with you, but with just the API I’m not sure if I will reach my goal: Fully automated provisioning. I could use Service Accounts to create data sources per API, but these are limited to the organization they belong to. So I would need to automatically create service accounts. Also for this, there exists an API, but how do I authenticate here? If if I would have, since service accounts are scoped, so I could only create new service accounts for the same organization. I’m missing the point in the documentation how I can actually build up an entire new org (with users, data sources, service accounts…) fully automated, not with user action.

1 Like
10

Use a hybrid approach

Lean heavily on API and provisioning where possible and something else where not

For a peosuct that leans heavily on a db with possible changing schemas per each release you cannot have fully automated but will end up with 10o% heavy user action

11

I guess you can use user with “superadmin” role and you can can provision resources in any org

Combine it with terraform Terraform Registry and you will have enterprise grade provisioning.