Variables are not populated unless manually updated

Hi,

I am running Grafana v9.1.0-73288pre (046a2602ff) in a Docker Container. For a while now I have set up a Dashboard to visualize and analyse my rock climbing training.

My Docker Container updates automatically using Watchtower. Since more recent updates (unfortunately, I cannot determine exactly which update) the automatic variable population when the dashboard is loaded does not work for me any more. In the past, I loaded the dashboard and the queries associated with the used variables would execute in the background and populate drop down menus, so variable values could be chosen to look into the climbing data. For example, I was able to choose between “all”, “climbing” or “bouldering” data when the dashboard was loaded. But when I load the dashboard now, the variable associated with the same drop-down menu has no values. This can be seen in the next screenshot.

When I have a look at the variables under dashboard settings, I can see that there are no values associated. When I manually hit the “update” button, the underlying query seems to be executed and the variable populated.

Afterwards, the associated drop-down menu is populated and the dashboard can be used as normal/before.

To populate all drop-down menus, I have to repeat the manual update for each associated variable. Then the whole dashboard is useable again.

My preliminary research did not come up with any solutions, or anyone else reported this or a similar problem. I have 2 thoughts on this (but please keep in mind that I really am a noob in this):

  1. One of the last Grafana updates “broke” something, which results in the variable queries not running automatically when a dashboard is loaded.

  2. There was maybe a setting introduced that the automatic execution of the variable queries has to be actived? Which I could not find, but maybe I am blind.

Any help or suggestions are highly appreciated, Thanks! :slight_smile:

@berwangerkris this isn’t expected, I have a couple of ideas you could try to repair the dashboard. The variables are stored in the database, so it’s possible something happened during a recent update.

  • What happens if you export the dashboard json and then import the dashboard? Use the toggle export for sharing externally, which will prompt you to select an input datasource when you import the dashboard - I suggest creating a new dashboard ID and name when you import so you can verify how it behaves.

@melori.arellano Thank you for the suggestion! :slight_smile:

I tried your export and import with new ID and name suggestion. I see the same behavior on the newly imported dashboard. When I load the dashboard, all variables are unpopulated. Only after I manually update the variables in settings, the variables (drop-down menus) are populated.

What is also weird: I have a variable var_grading, which is of type custom and lets me choose the grading system that the data in the dashboard should be converted to (climbing routes have different grading systems). According to the value of var_grading the variables var_from and var_to are populated. Both of these variables have exactely the same underlying query:

SELECT (CASE
WHEN ‘$var_grading’ = ‘french’ THEN grade_fr
WHEN ‘$var_grading’ = ‘uiaa’ THEN grade_uiaa
WHEN ‘$var_grading’ = ‘yds’ THEN grade_yds
WHEN ‘$var_grading’ = ‘font-scale’ THEN grade_fontscale
WHEN ‘$var_grading’ = ‘v-scale’ THEN grade_vscale
END)
FROM template_sorter

However, only var_to will be updated according to a change in var_grading. var_from only changes when I manually update through settings.

Is there a log I could look into to debug?

@berwangerkris I like to look at developer tools in the browser (the network tab) to see the requests and responses sent by the page (debug not required for this).

If you want to enable debug logging edit the configuration file grafana.ini:

[log]
# Either "console", "file", "syslog". Default is console and file
# Use space to separate multiple modes, e.g. "console file"
mode = console file

# Either "debug", "info", "warn", "error", "critical", default is "info"
level = debug

Then restart grafana for the setting to go into effect.

@melori.arellano I looked into the network tab of the dev tools in chrome. When loading the page, I can see some queries that are stated as status cancelled and there are errors present. These errors concern a Web Socket connection:

(I disguised my privat URL in the screenshot above)

I can export an HAR file. Would that help, or gives the screenshotted error enough information?


Edit: I saw a similar error description in this thread [Link]. Posters hinted that the correct root-URL has to be set under the [server] section in the Grafana config. I already had this URL set to my full domain (https://XYZ.ZYX.de/) for redirects to work properly. Also, I am running this Grafana Container on a Synology Diskstation and using the Synology built in Reverse Proxy. I thought I add this information for clarification, maybe.

What I also see in the network tab of the dev tools is the following query failures:

These failed queries appear when the website is loaded.

Also, when I select a value for a variable X in a drop-down menu, that should trigger a query for variable Y to repopulate the variable Y according to the value selected for variable X, I can see a failed query in the network tab dev tools:

When I navigate into the entry in dev tools under response, it says “Failed to load response data: No data found for resource with given identifier”. Which I find weird, because I can manually trigger those queries fine (as describe above), so the data is definitely there.