User Org permissions gone after logoff / logon (Azure Oauth2)

I’ve got a weird issue with our Grafana instance that i’m running locally. I’ve recently connected it to our Azure environment with Oauth2 (which is working), but Grafana doesn’t seem to remember the user permissions per org.

Example:
I give User “John” Admin access to Org 2 and 3. He refreshes his screen and they are visible.
As soon as he logs off and logs in again, the permissions are gone. He went back to having only Editor permissions on Org 1.

Can anybody help me in figuring this out? I’ve configured OAuth via this manual: https://grafana.com/docs/grafana/latest/auth/azuread/ (User John is in the App role “Editor”). I did look in the debug log, but there is nothing interesting there.

Thanks!