Grafana no longer showing metrics list from Prometheus data source - but data is visible

nurunet · February 25, 2024, 11:34am

Yesterday, I finished my first Grafana dashboard (monitoring connection quality metrics of my cable modem/router.

Everything is running in Docker. fritzbox_exporter fetches the data and provides them on port 9042 for Prometheus to pick up.

Prometheus does just that and shows fritzbox_exporter as a running, healthy Target. I can browse the metrics in Prometheus.

Prometheus is my default data source in Grafana. I adapted an imported dashboard to show data about modulation, error rates etc.

This worked dandy yesterday morning. Yesterday evening, I wanted to change one metric in a visualisation, and noticed two things.

For one, I got an error message:

The full text of one such message:

An unexpected error happened
Details
TypeError: Cannot read properties of undefined (reading ‘toLowerCase’)

at c (http://homelab.fritz.box:3000/public/build/1727.281a1e1cd15ef0881fa6.js:2:13549)
at div
at http://homelab.fritz.box:3000/public/build/8378.36d9cd16557e09d3c339.js:211:49620
at div
at B (http://homelab.fritz.box:3000/public/build/1727.281a1e1cd15ef0881fa6.js:2:13178)
at http://homelab.fritz.box:3000/public/build/1697.ba74e2de8c65e37f1de4.js:87:11436
at ia (http://homelab.fritz.box:3000/public/build/1697.ba74e2de8c65e37f1de4.js:87:13279)
at div
at m (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:26:25711)
at u (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:26:23378)
at http://homelab.fritz.box:3000/public/build/1697.ba74e2de8c65e37f1de4.js:98:4683
at Ta (http://homelab.fritz.box:3000/public/build/1697.ba74e2de8c65e37f1de4.js:98:6910)
at i (http://homelab.fritz.box:3000/public/build/8378.36d9cd16557e09d3c339.js:1:172)
at o (http://homelab.fritz.box:3000/public/build/8378.36d9cd16557e09d3c339.js:192:118841)
at g (http://homelab.fritz.box:3000/public/build/8378.36d9cd16557e09d3c339.js:192:119350)
at div
at div
at div
at fs (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:124:25127)
at It (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:187:4993)
at za (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:124:29667)
at Rs (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:124:29797)
at E (http://homelab.fritz.box:3000/public/build/8378.36d9cd16557e09d3c339.js:425:68)
at div
at K (http://homelab.fritz.box:3000/public/build/8378.36d9cd16557e09d3c339.js:4108:820)
at div
at mu (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:124:30083)
at It (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:187:4993)
at y (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:187:2689)
at Xo (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:124:12772)
at ft (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:66:950)
at Oa (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:124:14520)
at g (http://homelab.fritz.box:3000/public/build/8378.36d9cd16557e09d3c339.js:4109:6041)
at div
at div
at div
at div
at div
at q (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:124:36551)
at u (http://homelab.fritz.box:3000/public/build/8378.36d9cd16557e09d3c339.js:192:29691)
at H (http://homelab.fritz.box:3000/public/build/8378.36d9cd16557e09d3c339.js:4114:56)
at Hs (http://homelab.fritz.box:3000/public/build/DashboardPageProxy.309311965b4cd0612f76.js:315:9492)
at div
at r (http://homelab.fritz.box:3000/public/build/8378.36d9cd16557e09d3c339.js:284:49421)
at div
at http://homelab.fritz.box:3000/public/build/DashboardPageProxy.309311965b4cd0612f76.js:315:10940
at div
at div
at i (http://homelab.fritz.box:3000/public/build/8474.aca7493112eb29aea9e3.js:1:2301)
at div
at i (http://homelab.fritz.box:3000/public/build/8474.aca7493112eb29aea9e3.js:1:4289)
at _ (http://homelab.fritz.box:3000/public/build/8474.aca7493112eb29aea9e3.js:1:9608)
at div
at i (http://homelab.fritz.box:3000/public/build/8474.aca7493112eb29aea9e3.js:1:2301)
at div
at i (http://homelab.fritz.box:3000/public/build/8474.aca7493112eb29aea9e3.js:1:4289)
at _ (http://homelab.fritz.box:3000/public/build/8474.aca7493112eb29aea9e3.js:1:9608)
at div
at div
at div
at U (http://homelab.fritz.box:3000/public/build/8378.36d9cd16557e09d3c339.js:357:21347)
at Xs (http://homelab.fritz.box:3000/public/build/DashboardPageProxy.309311965b4cd0612f76.js:332:526)
at Ut (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:183:66326)
at WithTheme(Connect(Xs))
at vn (http://homelab.fritz.box:3000/public/build/DashboardPageProxy.309311965b4cd0612f76.js:407:474)
at DashboardPage
at Ut (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:183:66326)
at So (http://homelab.fritz.box:3000/public/build/DashboardPageProxy.309311965b4cd0612f76.js:407:6597)
at Suspense
at o (http://homelab.fritz.box:3000/public/build/8378.36d9cd16557e09d3c339.js:192:118841)
at Gc (http://homelab.fritz.box:3000/public/build/8378.36d9cd16557e09d3c339.js:5420:9842)
at ve (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:228:23578)
at la (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:209:4728)
at Zn (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:209:10219)
at el (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:228:11773)
at ve (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:228:25773)
at div
at div
at main
at div
at ag (http://homelab.fritz.box:3000/public/build/8378.36d9cd16557e09d3c339.js:5420:2407)
at la (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:209:4728)
at Zn (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:209:10219)
at _n (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:209:9608)
at Il (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:228:12072)
at ve (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:228:19749)
at div
at i (http://homelab.fritz.box:3000/public/build/8378.36d9cd16557e09d3c339.js:212:13888)
at v (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:41:14038)
at p (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:662:233850)
at c (http://homelab.fritz.box:3000/public/build/8378.36d9cd16557e09d3c339.js:566:21348)
at o (http://homelab.fritz.box:3000/public/build/8378.36d9cd16557e09d3c339.js:192:118841)
at g (http://homelab.fritz.box:3000/public/build/8378.36d9cd16557e09d3c339.js:192:119350)
at Re (http://homelab.fritz.box:3000/public/build/9733.976a8afd1c8ec137399a.js:187:529)
at Eu (http://homelab.fritz.box:3000/public/build/8378.36d9cd16557e09d3c339.js:5431:122)

Refreshing the page (F5) temporarily makes the error message disappear. However, I cannot pick a new metric. The drop-down says “Loading options…” and shows a spinning indicator, but the options are not loaded. Yesterday morning, this still worked.

I am puzzled, because the already configured metrics seem to keep updating as expected. Also, the metrics browser still shows them.

Any hints how I could try to troubleshoot?

yosiasz · February 25, 2024, 11:46am

Welcome @nurunet to forum

Which dashboard is it. Also what version of grafana

nurunet · February 25, 2024, 12:02pm

Thanks for your answer. The dashboard is from a magazine (c’t). They provided it for download here: https://ftp.heise.de/ct/listings/2022/19/1922_138.zip

I used “Dashboard3-Kabelmodem Leitungsqualität.json”.

That said, even when creating a new dashboard and adding a first visualisation, I get the spinning wheel/no metrics loading issue in the Builder.

I am running Grafana v10.3.3 (252761264e)

johnsmithfinch · February 26, 2024, 9:47am

I am getting the same error when trying to add a new dashboard:
An unexpected error happened
Details
TypeError: Cannot read properties of undefined (reading ‘toLowerCase’)

at p (https://grafana2:3000/public/build/7530.1906d7824537abe9101c.js:1:12435)
at div
at vt (https://grafana2:3000/public/build/4067.67791818270ded8beee8.js:40:275)
at div
at M (https://grafana2:3000/public/build/7530.1906d7824537abe9101c.js:1:12059)
at https://grafana2:3000/public/build/prometheusPlugin.5612d73c2fcce631047b.js:109:2682
at Ca (https://grafana2:3000/public/build/prometheusPlugin.5612d73c2fcce631047b.js:114:215)
at div
at vt (https://grafana2:3000/public/build/4067.67791818270ded8beee8.js:40:275)
at g.EditorRows (https://grafana2:3000/public/build/4067.67791818270ded8beee8.js:40:4973)
at https://grafana2:3000/public/build/prometheusPlugin.5612d73c2fcce631047b.js:125:4740
at Ra (https://grafana2:3000/public/build/prometheusPlugin.5612d73c2fcce631047b.js:125:6951)
at l (https://grafana2:3000/public/build/8674.32df99c8f0ee33b06f9f.js:1:172)
at m (https://grafana2:3000/public/build/8674.32df99c8f0ee33b06f9f.js:1278:180)
at c (https://grafana2:3000/public/build/8674.32df99c8f0ee33b06f9f.js:1278:754)
at div
at div
at div
at Pc (https://grafana2:3000/public/build/4067.67791818270ded8beee8.js:136:25176)
at An (https://grafana2:3000/public/build/4067.67791818270ded8beee8.js:174:126969)
at Ps (https://grafana2:3000/public/build/4067.67791818270ded8beee8.js:136:29742)
at es (https://grafana2:3000/public/build/4067.67791818270ded8beee8.js:136:29872)
at u (https://grafana2:3000/public/build/8674.32df99c8f0ee33b06f9f.js:5075:68)
at div
at M (https://grafana2:3000/public/build/8674.32df99c8f0ee33b06f9f.js:10032:682)
at div
at ws (https://grafana2:3000/public/build/4067.67791818270ded8beee8.js:136:30172)
at An (https://grafana2:3000/public/build/4067.67791818270ded8beee8.js:174:126969)
at S (https://grafana2:3000/public/build/4067.67791818270ded8beee8.js:174:124664)
at pc (https://grafana2:3000/public/build/4067.67791818270ded8beee8.js:136:12809)
at Je (https://grafana2:3000/public/build/4067.67791818270ded8beee8.js:78:961)
at Pu (https://grafana2:3000/public/build/4067.67791818270ded8beee8.js:136:14524)
at P (https://grafana2:3000/public/build/5106.e90fde3e471c14e62263.js:165:622)

I am running Version 10.3.1, I tried upgrade/downgrade, same error.

Any ideeas?

Thanks.

johnsmithfinch · February 26, 2024, 2:32pm

it was prometheus, I downgraded to 2.45 and now all good

nurunet · February 26, 2024, 3:11pm

I pulled 2.45.3 to try, but unfortunately that didn’t solve it.

johnsmithfinch · February 27, 2024, 11:04pm

Make a backup of your prometheus data and start with a clean database when downgrading to 2.45.3

jangaraj · February 28, 2024, 12:13am