Live API errors

Grafana Authentication
, ,
1

For the past week or so, I’ve been seeing occasional “unauthorized” messages in the Grafana webpage, and this line in the log:

logger=context userId=0 orgId=0 uname= t=2023-12-02T17:32:43.894792407Z level=info msg=“Request Completed” method=GET path=/api/live/ws status=401 remote_addr=172.24.80.239 time_ms=0 duration=676.986µs size=40 referer= handler=/api/live/ws

I think this may have started when I enabled https and OAuth in the Grafana config, but I’m not completely sure. Is there some additional config required to make this “live” websocket connection work? I’m not even using Grafana Live so I wouldn’t mind simply disabling it too if I could find a way to do that.

I’ve configured the following environment variables in Docker which may be relevant. (Redacted for privacy.)

 - GF_SERVER_CERT_FILE=
 - GF_SERVER_CERT_KEY=
 - GF_SERVER_DOMAIN=
 - GF_SERVER_ENFORCE_DOMAIN=false
 - GF_SERVER_PROTOCOL=https
 - GF_SERVER_ROOT_URL=
 - GF_SECURITY_COOKIE_SECURE=true
 - GF_AUTH_DISABLE_LOGIN_FORM=true
 - GF_AUTH_GENERIC_OAUTH_ALLOW_SIGN_UP=true
 - GF_AUTH_GENERIC_OAUTH_API_URL=
 - GF_AUTH_GENERIC_OAUTH_AUTH_URL=
 - GF_AUTH_GENERIC_OAUTH_CLIENT_ID=
 - GF_AUTH_GENERIC_OAUTH_CLIENT_SECRET__FILE=
 - GF_AUTH_GENERIC_OAUTH_ENABLED=true
 - GF_AUTH_GENERIC_OAUTH_SCOPES=email openid profile
 - GF_AUTH_GENERIC_OAUTH_SKIP_ORG_ROLE_SYNC=true
 - GF_AUTH_GENERIC_OAUTH_TOKEN_URL=
1 Like
2

Dont fret too nuch, its just informational

3

I don’t care so much about the log file but it’s showing a yellow error message pop up in the webpage that is annoying.

4

We cant see your screen remotely please post a screen grab and the actual error in the browser. It could be unrelated to the log error

5

Here’s the error, shown in the top right.

Screenshot 2023-12-02 at 2.29.25 PM
Screenshot 2023-12-02 at 2.29.25 PM2981×1199 491 KB

It seems like there are 2 endpoints causing similar errors, /api/live/publish and /api/live/ws.

logger=context userId=0 orgId=0 uname= t=2023-12-02T22:30:29.637945449Z level=info msg=“Request Completed” method=GET path=/api/live/publish status=401 remote_addr=172.24.80.239 time_ms=0 duration=632.281µs size=40 referer= handler=notfound

6

What is this netflix stuff?

Where are you hosting your grafana?

7

It’s self hosted in a Docker container.

8

bumping up this thread

9

I came here looking for an answer as Grafana behind a reverse proxy shows the same log entries.

2024/09/26 19:30:43 [error] 893292#893292: *4376 connect() failed (111: Connection refused) while connecting to upstream, client: 10.2.0.2, server: stats.domain.com, request: "GET /api/live/ws HTTP/1.1", upstream: "https://10.2.0.22:443/api/live/ws", host: "stats.domain.com"

So despite this being logged as informational, the Grafana instance does refuse to reply to a client requesting this URI. So this looks like an artifact left over from Grafana Cloud code which doesn’t translate to OSS deployments. I assume the URI comes from the server and the clients just try to fetch the resource.

10

Proxy needs explicit support/configuration for websockets. API live requests are not standard http calls.

1 Like
11

I thought that having ssl negated most of that, then added the following line to the reverse proxy on the Grafana host. But had to also add it to the second tier of reverse proxies… Duh/Doh, lesson learned (I hope).

proxy_http_version 1.1;

12

This is not useful, when you don’t say, which reverse proxy you have. I guess it’s Nginx. There is doc for Nginx and it mentions required config for live API:

1 Like
13

Apologies, you’re right. Nginx. I had everything in that suggested link except for the line I mentioned.