Hello,
I’m using grafana 9.2 docker image with grafana renderer latest docker image.
But rendering does not work because of the certificate used for grafana.
We get the following error in the logging:
{“failure”:“net::ERR_CERT_AUTHORITY_INVALID”,“level”:“error”,“message”:“Browser request failed”,“method”:“GET”,“url”:"https://grafana-url:3000
We use the following environment variables on grafana:
- GF_RENDERING_SERVER_URL=http://renderer:8081/render
- GF_RENDERING_CALLBACK_URL=https://grafana-url:3000/
Could someone point me to the environment variable for ignoring https errors?
Thanks in advance,
Ronald
I have the same issue, any one can help on that.
Hi @ronaldbuffing,
Welcome to the 
community support forums !!
Please check the following documentation which describes the troubleshooting steps for the Image Render Plugin:
I hope this helps.
Thank you for the link but I think the troubleshooting steps don’t apply to the docker image for the image-renderer!
Please try to increase the log level and check the docker logs so that can view what the complete error message is about and it could help for further investigation.
Here are the logs:
Image-renderer docker:
{"err":"Error: net::ERR_CERT_AUTHORITY_INVALID at [https ://x.x.x.x:3000/d-solo/mvrzTKxnk/scapacity-management?orgId=1&refresh=5m&from=1651063990218&to=1666871590218&panelId=21&width=1000&height=500&tz=Europe%2FAmsterdam&render=1\n](https ://x.x.x.x.x:3000/d-solo/mvrzTKxnk/sbt-capacity-management?orgId=1&refresh=5m&from=1651063990218&to=1666871590218&panelId=21&width=1000&height=500&tz=Europe%2FAmsterdam&render=1%5Cn) at navigate (/usr/src/app/node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:156:23)\n at processTicksAndRejections (node:internal/process/task_queues:96:5)\n at async FrameManager.navigateFrame (/usr/src/app/node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:131:21)\n at async Frame.goto (/usr/src/app/node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:512:16)\n at async Page.goto (/usr/src/app/node_modules/puppeteer/lib/cjs/puppeteer/common/Page.js:1167:16)\n at async Browser.takeScreenshot (/usr/src/app/build/browser/browser.js:256:13)\n at async Browser.render (/usr/src/app/build/browser/browser.js:230:20)\n at async HttpServer.render (/usr/src/app/build/service/http-server.js:53:28)","level":"error","message":"Error while trying to prepare page for screenshot","url":https://x.x.x.x:3000/d-solo/mvrzTKxnk/sbt-capacity-management?orgId=1&refresh=5m&from=1651063990218&to=1666871590218&panelId=21&width=1000&height=500&tz=Europe%2FAmsterdam&render=1}
{"err":"TimeoutError: waiting for function failed: timeout 60000ms exceeded\n at new WaitTask (/usr/src/app/node_modules/puppeteer/lib/cjs/puppeteer/common/DOMWorld.js:528:34)\n at DOMWorld.waitForFunction (/usr/src/app/node_modules/puppeteer/lib/cjs/puppeteer/common/DOMWorld.js:479:26)\n at Frame.waitForFunction (/usr/src/app/node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:1010:32)\n at Page.waitForFunction (/usr/src/app/node_modules/puppeteer/lib/cjs/puppeteer/common/Page.js:2490:33)\n at /usr/src/app/build/browser/browser.js:284:29\n at Browser.withTimingMetrics (/usr/src/app/build/browser/browser.js:411:20)\n at Browser.takeScreenshot (/usr/src/app/build/browser/browser.js:280:24)\n at processTicksAndRejections (node:internal/process/task_queues:96:5)\n at async Browser.render (/usr/src/app/build/browser/browser.js:230:20)\n at async HttpServer.render (/usr/src/app/build/service/http-server.js:53:28)","level":"error","message":"Error while waiting for the panels to load","url":https://x.x.x.x.x:3000/d-solo/mvrzTKxnk/sbt-capacity-management?orgId=1&refresh=5m&from=1651063990218&to=1666871590218&panelId=21&width=1000&height=500&tz=Europe%2FAmsterdam&render=1}
Grafana docker:
logger=rendering renderer=http t=2022-10-31T07:56:18.172371415Z level=debug msg="calling remote rendering service" url=http ://renderer:8081/render/version
logger=cleanup t=2022-10-31T07:56:18.181126431Z level=info msg="Completed cleanup jobs" duration=8.163771ms
logger=rendering renderer=http t=2022-10-31T08:05:21.464978824Z level=info msg=Rendering path="d-solo/AzGbq4VVz/lon-capacity-management?orgId=1&refresh=5m&from=1666595123867&to=1667203523867&panelId=61&width=1000&height=500&tz=Europe%2FAmsterdam"
logger=rendering renderer=http t=2022-10-31T08:05:21.48677008Z level=debug msg="calling remote rendering service" url=http ://renderer:8081/render?deviceScaleFactor=1.000000&domain=x.x.x.x.x&encoding=&height=500&renderKey=J2LEKGMu424KUF98m60x0MdALbwaX13U&timeout=60&timezone=Europe%2FAmsterdam&url=https%3A%2F%2Fx.x.x.x.x%3A3000%2Fd-solo%2FAzGbq4VVz%2Flon-capacity-management%3ForgId%3D1%26refresh%3D5m%26from%3D1666595123867%26to%3D1667203523867%26panelId%3D61%26width%3D1000%26height%3D500%26tz%3DEurope%252FAmsterdam%26render%3D1&width=1000
server.go:3230: http: TLS handshake error from [192.168.64.1:51812](http ://192.168.64.1:51812/): remote error: tls: unknown certificate
Hi @ronaldbuffing ,
Try to add this option in the docker-compose file as it might be helpful to resolve it.
GF_RENDERER_PLUGIN_IGNORE_HTTPS_ERRORS=true
Let us know the results.
I already have this environment variable set for the grafana docker image but this does not resolve the issue.
environment:
- GF_RENDERER_PLUGIN_IGNORE_HTTPS_ERRORS=true
- GF_PATHS_CONFIG=/var/lib/grafana/grafana.ini
- GF_RENDERING_SERVER_URL=http://renderer:8081/render
- GF_RENDERING_CALLBACK_URL=https://x.x.x.x:3000/
- GF_LOG_FILTERS=rendering:debug
Thanks for the quick reply.
Looking again at the attached logs you provided, it says:
server.go:3230: http: TLS handshake error from [192.168.64.1:51812](http ://192.168.64.1:51812/): remote error: tls: unknown certificate
So the error comes from the server with IP [192.168.64.1:51812] when trying to do a handshake.
To me it sounds like an issue with the certificate validity. I google around and found many related posts and threads but this one seems to be giving good information. Therefore please check and hopefully you may find the root cause and provide the solution to this post for other involved users.
There is nothing wrong with the certificate.
The ip is the ip of the grafana-image-renderer docker .
Setting the environment variable should ignore the https errors.
Please involve the grafana-image-renderer developer for this issue.
Hi @ronaldbuffing,
Could you please provide me with the complete docker-compse.yml and the version you used before?
Also, can you please provide as how you configure your SSL cert (i guess it is defined in the docker-compose file)?
Thanks
Below my docker-compose file:
grafana:
restart: always
image: grafana/grafana-enterprise:9.2.0
volumes:
- /opt/docker/metrics/grafana:/var/lib/grafana
ports:
- "3000:3000"
links:
- influxdb
depends_on:
- influxdb
environment:
- GF_RENDERING_IGNORE_HTTPS_ERRORS=true
- GF_RENDERER_PLUGIN_IGNORE_HTTPS_ERRORS=true
- GF_PLUGIN_GRAFANA_IMAGE_RENDERER_RENDERING_IGNORE_HTTPS_ERRORS=true
- GF_PATHS_CONFIG=/var/lib/grafana/grafana.ini
- GF_RENDERING_SERVER_URL=http://renderer:8081/render
- GF_RENDERING_CALLBACK_URL=https://xxxx:3000/
- GF_LOG_FILTERS=rendering:debug
renderer:
image: grafana/grafana-image-renderer:latest
ports:
- "8081"
Hi @ronaldbuffing,
Thanks for pasting the docker-compose.yml file.
Please move this environment variable:
- GF_RENDERING_IGNORE_HTTPS_ERRORS=true
from grafana into the renderer section and see if that helps.
I had the same issue with my Docker Grafana and Docker image renderer as well.
Instead of using GF_RENDERING_IGNORE_HTTPS_ERRORS=true on Grafana environment, configure it that way on the image render section:
renderer:
image: grafana/grafana-image-renderer:2.0.1
container_name: renderer
ports:
- "8081:8081"
networks:
- monitor-net
labels:
org.label-schema.group: "monitoring"
environment:
- IGNORE_HTTPS_ERRORS=true
restart: unless-stopped
Make sure that https_addr variable on grafana.ini is the same as your Grafana URL, same for the callback_url of renderer.
Best regards,
Bar.
4 Likes
hi, I am facing the same issue. would appreciate help with this.
grafana - 7.5.3
image render plugin - 3.5.0
here is how my docker_compose.yml
version: '3.7'
services:
grafana_7_3001:
image: grafana/grafana:7.5.3
restart: always
network_mode: bridge
volumes:
- ./config/defaults.ini:/usr/share/grafana/conf/defaults.ini:z
- ./config/provisioning:/etc/grafana/provisioning:z
- ./config/scripted_dashboards:/usr/share/grafana/public/dashboards:z
- ./config/grafana/dashboards:/var/lib/grafana/dashboards:z
- ./config/ssl:/var/lib/grafana/ssl:z
- ./config/grafana.db:/var/lib/grafana/grafana.db:z
ports:
- ${grafana7_port}:3000
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
environment:
- http_proxy=
- https_proxy=
- no_proxy=localhost,127.0.0.0/8,::1,.corp.xxxx.com,localhost,127.0.0.0/8,::1,.corp.xxxx.com,.corp.xxxx.com,10.236.196.82
- GF_RENDERING_SERVER_URL=http://10.236.196.82:9081/render
- GF_RENDERING_CALLBACK_URL=https://10.236.196.82:3001/
- GF_LOG_FILTERS=rendering:debug
- GF_RENDERER_PLUGIN_IGNORE_HTTPS_ERRORS=true
- GF_PLUGIN_GRAFANA_IMAGE_RENDERER_RENDERING_IGNORE_HTTPS_ERRORS=true
renderer:
image: grafana/grafana-image-renderer:3.5.0
restart: always
network_mode: bridge
ports:
- 9081:8081
environment:
- ENABLE_METRICS=true
- GF_RENDERING_IGNORE_HTTPS_ERRORS=true
i see following errors:
in grafana container logs
t=2022-12-28T09:05:42+0000 lvl=info msg=Rendering logger=rendering renderer=http path="d-solo/fnApLhaMz/alerts_status?orgId=1&from=1656407133100&to=1672218333100&panelId=3&width=1000&height=500&tz=Asia%2FCalcutta"
t=2022-12-28T09:05:42+0000 lvl=dbug msg="calling remote rendering service" logger=rendering renderer=http url="http://10.236.196.82:9081/render?deviceScaleFactor=1.000000&domain=10.236.196.82&encoding=&height=500&renderKey=Vwy54pbGa7OuOgq6KqUXrdc9Qw9tJls3&timeout=60&timezone=Asia%2FCalcutta&url=https%3A%2F%2F10.236.196.82%3A3001%2Fd-solo%2FfnApLhaMz%2Falerts_status%3ForgId%3D1%26from%3D1656407133100%26to%3D1672218333100%26panelId%3D3%26width%3D1000%26height%3D500%26tz%3DAsia%252FCalcutta%26render%3D1&width=1000"
2022/12/28 09:05:43 http: TLS handshake error from 172.17.0.1:53032: remote error: tls: unknown certificate
t=2022-12-28T09:06:43+0000 lvl=warn msg="Failed to close file" logger=rendering renderer=http path=/var/lib/grafana/png/V8Mm75bB2FkW8ac4bCZq.png err="close /var/lib/grafana/png/V8Mm75bB2FkW8ac4bCZq.png: file already closed"
in the image renderer container logs -
{"failure":"net::ERR_CERT_AUTHORITY_INVALID","level":"error","message":"Browser request failed","method":"GET","url":"https://10.236.196.82:3001/d-solo/fnApLhaMz/alerts_status?orgId=1&from=1656407133100&to=1672218333100&panelId=3&width=1000&height=500&tz=Asia%2FCalcutta&render=1"}
{"err":"Error: net::ERR_CERT_AUTHORITY_INVALID at https://10.236.196.82:3001/d-solo/fnApLhaMz/alerts_status?orgId=1&from=1656407133100&to=1672218333100&panelId=3&width=1000&height=500&tz=Asia%2FCalcutta&render=1\n at navigate (/usr/src/app/node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:156:23)\n at processTicksAndRejections (internal/process/task_queues.js:95:5)\n at async FrameManager.navigateFrame (/usr/src/app/node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:131:21)\n at async Frame.goto (/usr/src/app/node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:512:16)\n at async Page.goto (/usr/src/app/node_modules/puppeteer/lib/cjs/puppeteer/common/Page.js:1167:16)\n at async Browser.takeScreenshot (/usr/src/app/build/browser/browser.js:256:13)\n at async Browser.render (/usr/src/app/build/browser/browser.js:230:20)\n at async HttpServer.render (/usr/src/app/build/service/http-server.js:51:28)","level":"error","message":"Error while trying to prepare page for screenshot","url":"https://10.236.196.82:3001/d-solo/fnApLhaMz/alerts_status?orgId=1&from=1656407133100&to=1672218333100&panelId=3&width=1000&height=500&tz=Asia%2FCalcutta&render=1"}
{"err":"TimeoutError: waiting for function failed: timeout 60000ms exceeded\n at new WaitTask (/usr/src/app/node_modules/puppeteer/lib/cjs/puppeteer/common/DOMWorld.js:528:34)\n at DOMWorld.waitForFunction (/usr/src/app/node_modules/puppeteer/lib/cjs/puppeteer/common/DOMWorld.js:479:26)\n at Frame.waitForFunction (/usr/src/app/node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:1010:32)\n at Page.waitForFunction (/usr/src/app/node_modules/puppeteer/lib/cjs/puppeteer/common/Page.js:2479:33)\n at /usr/src/app/build/browser/browser.js:284:29\n at Browser.withTimingMetrics (/usr/src/app/build/browser/browser.js:396:20)\n at Browser.takeScreenshot (/usr/src/app/build/browser/browser.js:280:24)\n at processTicksAndRejections (internal/process/task_queues.js:95:5)\n at async Browser.render (/usr/src/app/build/browser/browser.js:230:20)\n at async HttpServer.render (/usr/src/app/build/service/http-server.js:51:28)","level":"error","message":"Error while waiting for the panels to load","url":"https://10.236.196.82:3001/d-solo/fnApLhaMz/alerts_status?orgId=1&from=1656407133100&to=1672218333100&panelId=3&width=1000&height=500&tz=Asia%2FCalcutta&render=1"}
Try to add one more variable in your environment section for renderer i.e.
environment:
- IGNORE_HTTPS_ERRORS=true
- ENABLE_METRICS=true
- GF_RENDERING_IGNORE_HTTPS_ERRORS=true
then re-up your docker-compose with the --force-recreate flag i.e.
docker-compose up -d --force-recreate
1 Like
thanks @usman.ahmad . I saw that inside the grafana image render plugin we have this config.json file. and i updated the property “ignoresHttpsErrors” to true there. and it worked. I believe the above setting you mentioned IGNORE_HTTPS_ERRORS=true, will have the same effect.
thanks!
1 Like
hi - follow up question if anyone can help.
I was able to make it work by having a separate docker container for the image renderer.
However, when I try with custom docker image where the grafana-image-render is pre-installed, I am again getting the same ERR_CERT_AUTHORITY_INVALID error as below -
is there a way I can set the env variables for grafana and image-render inside the custom Dockerfile, like we did above in the docker-compose file ?
t=2023-01-05T09:11:00+0000 lvl=info msg=Rendering logger=rendering renderer=plugin path="d-solo/Nrr-mqpVk/atomiq-diagnostics-kpi?orgId=1&refresh=1m&from=1672737056662&to=1672909856662&panelId=48&width=1000&height=500&tz=Asia%2FCalcutta"
2023/01/05 09:11:01 http: TLS handshake error from [::1]:57782: remote error: tls: unknown certificate
t=2023-01-05T09:11:01+0000 lvl=eror msg="Browser request failed" logger=plugins.backend pluginId=grafana-image-renderer url="https://community.grafana.com/d-solo/Nrr-mqpVk/atomiq-diagnostics-kpi?orgId=1&refresh=1m&from=1672737056662&to=1672909856662&panelId=48&width=1000&height=500&tz=Asia%2FCalcutta&render=1" method=GET failure=net::ERR_CERT_AUTHORITY_INVALID
t=2023-01-05T09:11:01+0000 lvl=eror msg="Error while trying to prepare page for screenshot" logger=plugins.backend pluginId=grafana-image-renderer url="https://community.grafana.com/d-solo/Nrr-mqpVk/atomiq-diagnostics-kpi?orgId=1&refresh=1m&from=1672737056662&to=1672909856662&panelId=48&width=1000&height=500&tz=Asia%2FCalcutta&render=1" err="Error: net::ERR_CERT_AUTHORITY_INVALID at https://community.grafana.com/d-solo/Nrr-mqpVk/atomiq-diagnostics-kpi?orgId=1&refresh=1m&from=1672737056662&to=1672909856662&panelId=48&width=1000&height=500&tz=Asia%2FCalcutta&render=1\n at navigate (/snapshot/grafana-image-renderer/node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:156:23)\n at processTicksAndRejections (internal/process/task_queues.js:95:5)\n at async FrameManager.navigateFrame (/snapshot/grafana-image-renderer/node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:131:21)\n at async Frame.goto (/snapshot/grafana-image-renderer/node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:512:16)\n at async Page.goto (/snapshot/grafana-image-renderer/node_modules/puppeteer/lib/cjs/puppeteer/common/Page.js:1167:16)\n at async Browser.takeScreenshot (/snapshot/grafana-image-renderer/build/browser/browser.js:256:13)\n at async Browser.render (/snapshot/grafana-image-renderer/build/browser/browser.js:230:20)\n at async PluginGRPCServer.render (/snapshot/grafana-image-renderer/build/plugin/v2/grpc_plugin.js:107:13)"
t=2023-01-05T09:12:01+0000 lvl=eror msg="Error while waiting for the panels to load" logger=plugins.backend pluginId=grafana-image-renderer err="TimeoutError: waiting for function failed: timeout 60000ms exceeded\n at new WaitTask (/snapshot/grafana-image-renderer/node_modules/puppeteer/lib/cjs/puppeteer/common/DOMWorld.js:528:34)\n at DOMWorld.waitForFunction (/snapshot/grafana-image-renderer/node_modules/puppeteer/lib/cjs/puppeteer/common/DOMWorld.js:479:26)\n at Frame.waitForFunction (/snapshot/grafana-image-renderer/node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:1010:32)\n at Page.waitForFunction (/snapshot/grafana-image-renderer/node_modules/puppeteer/lib/cjs/puppeteer/common/Page.js:2479:33)\n at /snapshot/grafana-image-renderer/build/browser/browser.js:284:29\n at Browser.withTimingMetrics (/snapshot/grafana-image-renderer/build/browser/browser.js:396:20)\n at Browser.takeScreenshot (/snapshot/grafana-image-renderer/build/browser/browser.js:280:24)\n at processTicksAndRejections (internal/process/task_queues.js:95:5)\n at async Browser.render (/snapshot/grafana-image-renderer/build/browser/browser.js:230:20)\n at async PluginGRPCServer.render (/snapshot/grafana-image-renderer/build/plugin/v2/grpc_plugin.js:107:13)" url="https://community.grafana.com/d-solo/Nrr-mqpVk/atomiq-diagnostics-kpi?orgId=1&refresh=1m&from=1672737056662&to=1672909856662&panelId=48&width=1000&height=500&tz=Asia%2FCalcutta&render=1"
Hi @utsukprani,
Can you please provide the steps of reproduction so that the community can also try to reproduce it on a test machine?
Thanks.
hi @usman.ahmad -
I took the following Dockerfile from grafana -
Dockerfile
As I am using Grafana 7.5.3, I changed the --pluginUrl to ‘https://github.com/grafana/grafana-image-renderer/releases/download/v3.5.0/plugin-linux-x64-glibc-no-chromium.zip’. This is because as per the documentation of Grafana Image render, v3.5.0 is compatible with 7.5.
Now when I try to run my grafana image rendering, it gives the above ERR_CERT_AUTHORITY_INVALID error.
Now earlier when I used the docker-compose and separate container for image render, I was able to overcome this cert issue by using the ENV variable -
environment:
- IGNORE_HTTPS_ERRORS=true
- ENABLE_METRICS=true
- GF_RENDERING_IGNORE_HTTPS_ERRORS=true
I am not sure how can i set these ENV variables to the renderer plugin in the custom image via the Dockerfile.
thanks for the help !
