Ansible installing dashboards getting error - Templating Failed to upgrade legacy queries

Grafana Dashboards
1

I’m using OSS version 10.2.3 and I have exported the templates like they should be; however, I’m using ansible to deploy the dashboard to a production environment where the end user cannot modify the dashboard and should note be able too for contract reasons.

The problem is that when they are imported I get the error because in the Variables section of the bashboard the Query option datasource is not populated. But when I select the correct one I get an error: “This dashboard cannot be saved from the Grafana UI because it has been provisioned from another source.” Which is expected.

If I manually import the dashboard, which cannot be done in production and select the correct datasource for each of the variables everything works as it should.

The way I do the import with ansible is to copy the dashboard to the /opt/grafana-dashboard/VMware/dashboard.json location and copy a dashboard folder layout file to /etc/grafana/provisioning/dashboards/dashboard.yml, then restart the service and the folders are created and the dashboards are imported.

Anyway, I have the correct information for the datasources in the dashboard.json file, but it is not getting populated correctly:

 "__inputs": [
    {
      "name": "DS_INFLUXDBV2-FLUX-VMWARE",
      "label": "influxdbv2-flux-vmware",
      "description": "",
      "type": "datasource",
      "pluginId": "influxdb",
      "pluginName": "InfluxDB"
    },
    {
      "name": "DS_INFLUXDBV2-FLUX-LINUX",
      "label": "influxdbv2-flux-linux",
      "description": "",
      "type": "datasource",
      "pluginId": "influxdb",
      "pluginName": "InfluxDB"
    }
  ],

If I scroll down the in the file I see this:

      "datasource": {
        "type": "influxdb",
        "uid": "${DS_INFLUXDBV2-FLUX-VMWARE}"
      },
      ....................

I’m thinking it would pick up the correct datasource value, but it is not getting populated.

No sure where else in the dashboard.json file I would need to modify it so when it is imported the correct datasources are loaded or if I have to figure out another approach.

Thanks for any insight.

2

Hey @cstewa20,

Can you share your playbook?

Regards

3

Sure, it uses a lot of variables. I’m trying to work on a solution that will just put in the correct UID because when you look at the file when it tries to save it is changing.

"uid": "${DS_INFLUXDBV2-FLUX-VMWARE}"

To

"uid": "PBD94EFD07AE38705"

My thought would be to get the actual uid from the datasource when it is created, then populate that template configuration before coping the file to the correct location. However; running into an issue because in order to get the datasource UID I need to first get an Admin API token, which is a manual process for an administrator of the system to do and the end users don’t have that permission. Not sure why Grafana can’t allow you to set an api key in the ini file like they do with the admin password, it would make this much easier.

  • name: Set InfluxDB URL variable
    ansible.builtin.set_fact:
    influxdb_url: “{{ hostvars[groups[‘influxdb’][0]][‘ip_port’] }}”
    run_once: true

  • name: Template out influxdb flux datasource files
    ansible.builtin.template:
    src: influxdb.flux.conf.j2
    dest: “/etc/grafana/provisioning/datasources/influxdb-flux-{{ item.bucket }}.yml”
    group: grafana
    owner: grafana
    mode: 0640
    loop: “{{ influxdb_api_tokens }}”

  • name: Create Grafana directory
    ansible.builtin.file:
    path: “/opt/grafana-dashboards”
    state: directory
    group: grafana
    owner: grafana

  • name: Create directories for Grafana folders
    ansible.builtin.file:
    path: “/opt/grafana-dashboards/{{ item }}”
    state: directory
    group: grafana
    owner: grafana
    loop: “{{ grafana_dashboards | map(attribute=‘folder’) | unique | list }}”

  • name: Template out Grafana dashboard files
    ansible.builtin.template:
    src: “templates/{{ item.filename }}.j2”
    dest: “/opt/grafana-dashboards/{{ item.folder }}/{{ item.filename }}”
    group: grafana
    owner: grafana
    mode: 0640
    loop: “{{ grafana_dashboards }}”

  • name: Copy dashboard.yml to "/etc/grafana/provisioning/dashboad directory
    ansible.builtin.copy:
    src: “dashboard-structure.yml”
    dest: “/etc/grafana/provisioning/dashboards/dashboard.yml”
    group: grafana
    owner: grafana
    mode: 0640

  • name: Restart Grafana Server Service
    ansible.builtin.service:
    name: grafana-server
    state: restarted

4

I just want to be sure about your issue:
Is your problem that you can’t manage to dynamically populating the datasource UIDs in your dashboard JSON files?

5

Correct - the data source value is not getting populated

image
image544×567 23.3 KB

6

Okay, your issue is way too hard for my knowledge, but I assume you can’t ask Ansible to retrieve the token because of security issue.
I recommend you to create a topics about it on the Ansible community. Maybe you can find a workaround to retrieve the API token. Which would solve a big part of your problem…

Also, I just read in the documentation of Grafana that you can generate API key to authenticate applications with Grafana. I assume you could generate that with Ansible then use it to retrieve the good UID you need and continue with your playbook.
I have no idea if that could be working, just making some suggestion :neutral_face:…

Let me know how it goes!

Best regards

7

Thanks to taking the time to review. It seems this is a Grafana issue as it should work based on the documentation (Provision Grafana | Grafana documentation) (InfluxDB data source | Grafana documentation).

1 Like
8

The way I was finally about to get this to work is the following.

I had to manually create an API key with admin permissions, then

Then created this task to get the UID values:

- name: Get all datasources from Grafana
  uri:
    url: "http://{{grafana_url}}:3000/api/datasources"
    method: GET
    headers:
      Authorization: "Bearer {{ grafana_api_key }}"
    return_content: yes
    status_code: 200
  register: grafana_datasources

Then had to set the fact for those api keys; this required to have jmespath so (pip3 install jmespath - made that a task of course),

 - set_fact:
    influxdbv2_flux_linux_uid: "{{ grafana_datasources.json | json_query(query) }}"
    influxdbv2_flux_vmware_uid: "{{ grafana_datasources.json | json_query(query) }}"
  vars:
    query: "[?name=='influxdbv2-flux-vmware'].uid | [0]"

 - set_fact:
    influxdbv2_flux_linux_uid: "{{ grafana_datasources.json | json_query(query) }}"
    influxdbv2_flux_vmware_uid: "{{ grafana_datasources.json | json_query(query) }}"
  vars:
    query: "[?name=='influxdbv2-flux-linux'].uid | [0]"

Then modified all my dashboard with the correct variable:

"uid": "{{ influxdbv2_flux_vmware_uid }}"

Now at least the dashboards are working. Not sure how to make grafana work like the documents state, but this was a workaround. Still not sure how to get an admin key from the command line since my end user are just viewers and not admins. We make all the changes then deploy to the end user sites, so no admin on site.

Not sure how to contact the developers to see if this is a bug or if this is how you would have to do it.

1 Like
9

Hey @cstewa20,

Sounds good, can you modify your message and use the syntax ```yml for the code?

Regards

10

I think the docs are unclear. In order to provision a running instance through the API, you have to generate an API key with the correct role (RBAC might be in use for enterprise). But to do this, you have to login as an administrator to create the API key, then use it in your Ansible setup.

The docs are also describing how to drop in a “provisioning” directory that will be read on startup. You could use Ansible to do this too - git clone a repo into the provisioning directory and everything will be populated as expected when Grafana starts.

Dashboards can also be set on a scan interval while Grafana is running.

I think we need to revise the docs to reflect different scenarios (ansible/k8/cloud) so each method is given the right context.

1 Like
11

Correct this is the way I do it; however, it seems that the dashboard variables for the datastore are not getting set. This would be for an offline “no internet access” type of a network.

I need a way to have this work without first having to log in with an admin account to create the api key. When we deploy the dashboards we have users onsite, but no “admin” level people.

12

I’ll try this out and see if I can replicate that scenario.

The variables (if they are queries to an existing datasource) should populate depending on the dashboard configuration.