Grafana EntraID Oauth2 failing to get token

Hello all,

2 days ago out Grafana 9.5.5 stopped getting tokens when authenticating via Oauth2 EntraID, there were no changes to Grafana itself.

Grafana logs only following error in debug level:
logger=context userId=0 orgId=0 uname= t=2024-08-12T13:30:44.719227309Z level=error msg=login.OAuthLogin(NewTransportWithCode) error="Post \"https://login.microsoftonline.com/****/oauth2/v2.0/token\": re tcp ***:59136->***:443: read: connection reset by peer"

Grafana is able to get authorization code, whole process seems fine until POST stage to token endpoint.

Both authorize and token endpoint are reachable, here is also auth config:

    [auth.azuread]
    auth_url = https://login.microsoftonline.com/****/oauth2/v2.0/authorize
    client_id = $__file{/etc/secrets/CLIENT_ID}
    client_secret = $__file{/etc/secrets/CLIENT_SECRET}
    enabled = true
    name = Azure AD
    role_attribute_strict = false
    scopes = openid email profile
    token_url = https://login.microsoftonline.com/****/oauth2/v2.0/token
    allow_assign_grafana_admin = true
    skip_org_role_sync = false
    allow_sign_up = true
    auto_login = false
    allowed_domains=                                                                                                                                                                                                                          
    allowed_groups=

when I took authorization code obtained by grafana and POSTed it via wget from grafana container I got token, in EntraID there are no errors in Sign-in logs. Is somebody else facing same problems? Is there some way how to check what is Grafana posting to token endpoint?

Thanks a lot

Your Grafana doesn’t have proper network connectivity to login.microsoftonline.com:443 - ask your network team/support why.

I was able to get token via wget from same grafana container, so it doesn’t seem likely

Wget != Grafana and error:

Post \"https://login.microsoftonline.com/****/oauth2/v2.0/token\": re tcp ***:59136->***:443: read: connection reset by peer

saying that connection was not successful. wget may use proxy and Grafana may not - only you (should) know your network.

well yes, we are large company, would you know how to tshoot what grafana is sending in POST to token endpoint?

That is standard OAuth code for token exchange RFC 8693 - OAuth 2.0 Token Exchange

Large company, so: WAF, firewall, http proxy, deep packed inspection, … - many options, which may affect connectivity