Unexpected end of JSON input - Only when changing Time Range

Grafana Dashboards
1
  • What Grafana version and what operating system are you using?

Grafana V9.10-beta1
Running on Docker
Fronted by Traefik Proxy

  • What are you trying to achieve?

Dashboard of InfluxDB measurements

  • What happened?

When I change a variable of the time range, some of the panels get an “Unexpected end of JSON input” error and red exclamation. It is random which panel it happens to and how many. Sometimes it does not happen.

  • What did you expect to happen?

The dashboard load normally. When I “refresh” the browser page, it loads with no errors every time, no matter which time range is selected.

  • Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.

image
image

logger=context traceID=00000000000000000000000000000000 userId=1 orgId=1 uname=admin t=2022-08-13T18:14:53.386973415Z level=error msg="Request error" error="net/http: abort Handler"
logger=data-proxy-log userId=1 orgId=1 uname=admin path=/api/datasources/proxy/2/query remote_addr=10.1.20.43 referer="https://grafana.redacted.net/d/r5M0DLkVz/fcu-dashboard?orgId=1&from=now-2d&to=now&var-MachineID=All&var-MachineName=All&var-MachineContinent=All&var-Status=All" traceID=00000000000000000000000000000000 t=2022-08-13T18:21:45.595538303Z level=error msg="Proxy request error" error="httputil: ReverseProxy read error during body copy: read tcp 10.0.4.233:40036->10.0.4.71:8086: use of closed network connection"
logger=data-proxy-log userId=1 orgId=1 uname=admin path=/api/datasources/proxy/2/query remote_addr=10.1.20.43 referer="https://grafana.redacted.net/d/r5M0DLkVz/fcu-dashboard?orgId=1&from=now-2d&to=now&var-MachineID=All&var-MachineName=All&var-MachineContinent=All&var-Status=All" traceID=00000000000000000000000000000000 t=2022-08-13T18:05:51.03783699Z level=error msg="Proxy request error" error="httputil: ReverseProxy read error during body copy: read tcp 10.0.4.233:60366->10.0.4.71:8086: use of closed network connection"

From the error above, this seems like a traefik issue. “use of closed network connection”. Any advice on fixing?

2

Just bumping this… still an issue, there was a thread on the grafana reddit where some other users were experiencing this issue as well. Anyone have any input?

3

When it happens, capture the jsom data returned and copy paste it into notepad++ with json plugin installed and validate or any online json validator and validate there.
Could be the data source is returning an invalid json.

4

Thanks for your reply! I am able to capture lots of the JSON responses through dev console, but every one I check looks good according to an online JSON validator. So I’m not sure, maybe I’m not checking the right ones or I don’t know how to zero in on the one that caused the error specifically. I’ll keep trying…

The part that is puzzling is that even when the data doesn’t change, you can just wait for Grafana to refresh the panel and it will fix itself, then a few refreshes later might get the error again… so it’s totally random.

5

Could also be length of the json and what on grafana side it max json length
If for ex the json coming back is 1001 length and grafana or whatever json plugin you use to parse it does 1000 then you will get an error

I would do a python test of every json entry in your influxdb

6

Maybe when u change data range, does it happen when you change it manually or via calendar date picker?

7

it happens randomly when the dashboard refreshes itself and nothing changes. So I think that proves there isn’t a problem with the actual JSON from influxdb, doesn’t it?

8

I have changed the Database HTTP method from POST to GET and it solved the issue.
image