How to connect the Grafana AI Assistant to self-hosted Grafana instance

  • What Grafana version and what operating system are you using?
    I use grafana/grafana-enterprise:13.2.5 at docker/docker compose

  • What are you trying to achieve?
    Try to connect / configure the connection of Grafana AI Assistant on my self hosted Grafana

  • How are you trying to achieve it?
    I use the Quick connect button, and follow the wizzard. I’m loged in at my Grafana Cloud account too.

  • What happened?
    Quick feedback from wizzard, connection is successfully, but back at UI of my self hosted Grafana it shows, Connection failed:

    Connection failed
    
    Failed to check terms requirements: [object Object] Check your Backend URL and API Token below.
    
  • What did you expect to happen?

  • Can you copy/paste the configuration(s) that you are having problems with?

  • Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.

  • Did you follow any online instructions? If so, what is the URL?
    Yes, Set up Grafana Assistant in self-managed Grafana | Grafana Cloud documentation

  • I try the Manual configuration, but - how to find this information’s?:

    Backend URL, Instance ID and API Token.

    I dont know how if the Backend URL is right, I clear my Instance ID form screenshot for security - and I cant find the API token for the AI Service under Administration/Users and access/Cloud access policies under my Grafana Cloud Instance.

    Thanks for help…

Go to your Grafana Cloud account and your stack. There is Grafana Assistant section (https://grafana.com/orgs/<org>/grafana-assistant/<stack-id>) where you have details for Manual configuration, e.g.:

.

Check Grafana logs - there can some be useful insights about error, e.g. connection error, …

Thank you jangaraj,

After “Create token” → Error.

Log into my Grafana Cloud instance and

I delete the 9 tokens, and try again the Grafana Assistant Configuration

And it works, but error at self hosted grafana instance with this configuration.

My docker log:

logger=plugin.grafana-assistant-app level=error msg="error verifying idToken" endpoint=callResource err="invalid token: failed to parse as jwt token" pluginVersion=2.0.94
logger=context userId=1 orgId=1 uname=admin level=info msg="Request Completed" method=GET path=/api/plugins/grafana-assistant-app/resources/api/v1/settings/terms status=401 referer=https://grafana.example.com/a/grafana-assistant-app
logger=context userId=1 orgId=1 uname=admin level=info msg="Request Completed" method=POST path=/api/plugins/grafana-assistant-app/resources/api/v1/ofrep/v1/evaluate/flags status=401 referer=https://grafana.example.com/plugins/grafana-assistant-app?page=enable-assistant

“This repeats on every request (debug logging enabled). Grafana’s id-service also logs Sign new id token id=user:1, so signing seems to work.”

Also tried YAML provisioning (apps: grafana-assistant-app, ossMode: true, accessToken as “:”)
as described in thread 163046. Grafana loads it without errors (“Updating app from configuration
type=grafana-assistant-app enabled=true”), but the same “error verifying idToken … failed to parse
as jwt token” / 401 on /settings/terms and /ofrep/v1/evaluate/flags remains. It also fails on the
“Enable Assistant” page, so the connection settings are not involved.

Open your Grafana Cloud Grafana (not your local Grafana) and configure assistant there: Set up Grafana Assistant in Grafana Cloud | Grafana Cloud documentation

It looks like you didn’t accept terms and conditions for Grafana Assistant, so API requests are failing.

I accepted the Terms and Conditions months ago. In Grafana Cloud I can work with AI Assistant.

Hi @jmueller71

You mentioned Assistant works in Grafana Cloud, so the problem looks like it’s on the self hosted side. In your Docker log, error verifying idToken … failed to parse as jwt token comes from the plugin in your local Grafana.
The 401s on both /settings/terms and /ofrep/v1/evaluate/flags suggest a general authentication failure and not just the terms check.

To narrow it down, could you share →

the environment: section of your compose file and your root_url setting (secrets removed)

whether Grafana runs behind a reverse proxy (your log shows grafana.example.com)

the Grafana log lines from a fresh start that mention assistant or idToken

Also, please revoke the token shown in your screenshot (Cloud access policies → delete it) and create a new one. The glc_… value is readable in the image.

hi @jmueller71,

any chance you can share your docker-compose setup? also, are you using multiple orgs in your grafana setup?

also happy to chat on our community slack if that makes debugging easier, shoot me a message: Slack

Hi @jmueller71, thanks for the detailed logs, they enabled us to track this down!

What’s happening: your Grafana Cloud setup and Assistant configuration are fine. The problem is in your self-hosted Grafana. For every logged-in user, Grafana signs a short-lived ID token, and Assistant uses it to identify you. The key that signs those tokens is stored in Grafana’s
database, encrypted with Grafana’s secret_key. In your setup, that key was encrypted with a different secret_key than the one Grafana is running with now. Typical causes are a changed GF_SECURITY_SECRET_KEY, a grafana.ini that’s no longer mounted, or a database volume copied from
another setup. Grafana can’t decrypt the key, so it silently stops sending the ID token, and Assistant rejects the requests (error verifying idToken … failed to parse as jwt token). That’s also why you see Sign new id token on every request.

We can’t fix this from our side because the fix is in your local Grafana. You have two options:

Option A (recommended): restore the original secret key
If you know the secret_key you used before the upgrade, set it again in your compose file (GF_SECURITY_SECRET_KEY=…) and restart Grafana. This also fixes any other saved secrets, such as data source passwords.

Option B: reset the signing keys
If the old key is lost, back up the database and delete the signing keys. Grafana recreates them automatically, and they’re only used for short-lived tokens:

docker compose stop grafana
docker run --rm -v <your_grafana_volume>:/data alpine sh -c \
  'apk add -q sqlite && cp /data/grafana.db /data/grafana.db.bak && sqlite3 /data/grafana.db "DELETE FROM signing_key;"'
docker compose start grafana

Then reconnect Assistant (plugin → Connection → Cancel → Quick connect → “I’m reconnecting the same Grafana instance”) so its API token is saved again under the current key. Re-enter passwords for any data sources that stopped working.

To keep it from coming back: set the secret key explicitly in your compose file (or use GF_SECURITY_SECRET_KEY__FILE) and keep it with your backups. As long as the key doesn’t change, this is a one-time fix.

Unfortunately this was far too hard for users to diagnose. Grafana didn’t log anything useful, and the UI showed [object Object]. We’re going to improving both:
• Grafana will log a clear Failed to sign id token error that names the cause: Auth: Log ID token signing failures instead of silently forwarding no token by kevelopment · Pull Request #136408 · grafana/grafana · GitHub
• Assistant will show an explanatory message instead of [object Object] (PR merged, will be included in the next Release)

Important sidenote: Also, please revoke the glc_… token that’s visible in your earlier screenshot (Cloud access policies → delete) and create a new one.

Let us know if this helped you resolve the issue! :slight_smile:

Hi all,

thank you for your time.

Important sidenote: Also, please revoke the glc_… token that’s visible in your earlier screenshot (Cloud access policies → delete) and create a new one.

Yes I did 10sec after posting :wink:

@infofcc3 My posted log was cleaned by local AI chat.

In general, I can say that my Grafana server is running flawlessly; I’m collecting metrics and logs from my Linux fleet. Everything has been working for about 3 months—dashboards, alerts.

Ty

Thanks a lot, that was exactly it. Deleting the rows in signing_key (after changing GF_SECURITY_SECRET_KEY from
the default, which I did after the Advisor warned about it) fixed it. Assistant now shows “Connected” on
grafana-enterprise 13.3.0. Good to know for others: changing secret_key on an existing database makes the
ID-token signing key unreadable and Assistant fails silently with “error verifying idToken”. Thanks for the
quick fixes (log message + UI message), those will help a lot.

Greetings
Jens