MQTT data not stored

[michaelw]

• What Grafana version and what operating system are you using?
  Grafana v10.4.1 (d3ce857c0e)
  Debian GNU/Linux 12 (bookworm)
  Linux 6.6.20+rpt-rpi-v8 #1 SMP PREEMPT Debian 1:6.6.20-1+rpt1 (2024-03-07) aarch64 GNU/Linux

• What are you trying to achieve?
  I’m trying to build a dashboard for MQTT data.

• How are you trying to achieve it?
  I installed the plugin, added a data source (connection test ok) and I added a time series panel to my dashboard.

• What happened?
  Data is fetched properly and shown in the panel:
  

  

But once I navigate away from the dashboard the historic data is lost and it shows “no data”. Even if I just change the time range selection. Seems like the data is not stored properly. Not sure if this is an MQTT or a database issue?

• What did you expect to happen?
  I freshly installed Grafana and was hoping data is stored in the default sqlite db as per documentation without any additional manual steps on my side, isn’t it?

• Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.
  No error in the UI, except the “no data” info. Whenever I update the time range the grafana.log write these messages:

logger=live t=2024-04-05T13:35:50.900779495+02:00 level=info msg="Initialized channel handler" channel=ds/adhk0hz66yigwd/2s/plant_humidity_1 address=ds/adhk0hz66yigwd/2s/plant_humidity_1

Apart from that I just see cleanup and update entries in the log. Maybe this info during startup sounds suspicious:

Running in alternative execution of Error/NoData mode

→ NoData mode ? Unfortunately I couldn’t find information if this is relevant.

What is wrong here? Sorry if the reason might be obvious to you experts as the is actually my first Grafana experience.

Regards,
Michael

[michaelw]

Here’s the log:

logger=settings t=2024-04-05T13:25:16.941807153+02:00 level=info msg="Starting Grafana" version=10.4.1 commit=d3ce857c0eb86f571ffa993a9cd8493b6f47b630 branch=HEAD compiled=2024-04-05T13:25:16+02:00
logger=settings t=2024-04-05T13:25:16.94271601+02:00 level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini
logger=settings t=2024-04-05T13:25:16.942796824+02:00 level=info msg="Config loaded from" file=/etc/grafana/grafana.ini
logger=settings t=2024-04-05T13:25:16.942826379+02:00 level=info msg="Config overridden from command line" arg="default.paths.data=/var/lib/grafana"
logger=settings t=2024-04-05T13:25:16.942850656+02:00 level=info msg="Config overridden from command line" arg="default.paths.logs=/var/log/grafana"
logger=settings t=2024-04-05T13:25:16.942880989+02:00 level=info msg="Config overridden from command line" arg="default.paths.plugins=/var/lib/grafana/plugins"
logger=settings t=2024-04-05T13:25:16.94290584+02:00 level=info msg="Config overridden from command line" arg="default.paths.provisioning=/etc/grafana/provisioning"
logger=settings t=2024-04-05T13:25:16.94292947+02:00 level=info msg=Target target=[all]
logger=settings t=2024-04-05T13:25:16.942976117+02:00 level=info msg="Path Home" path=/usr/share/grafana
logger=settings t=2024-04-05T13:25:16.942998561+02:00 level=info msg="Path Data" path=/var/lib/grafana
logger=settings t=2024-04-05T13:25:16.943022839+02:00 level=info msg="Path Logs" path=/var/log/grafana
logger=settings t=2024-04-05T13:25:16.943066079+02:00 level=info msg="Path Plugins" path=/var/lib/grafana/plugins
logger=settings t=2024-04-05T13:25:16.94308756+02:00 level=info msg="Path Provisioning" path=/etc/grafana/provisioning
logger=settings t=2024-04-05T13:25:16.943113152+02:00 level=info msg="App mode production"
logger=sqlstore t=2024-04-05T13:25:16.94402425+02:00 level=info msg="Connecting to DB" dbtype=sqlite3
logger=migrator t=2024-04-05T13:25:16.945732909+02:00 level=info msg="Starting DB migrations"
logger=migrator t=2024-04-05T13:25:17.014026996+02:00 level=info msg="migrations completed" performed=0 skipped=548 duration=1.652105ms
logger=secrets t=2024-04-05T13:25:17.02019433+02:00 level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1
logger=plugin.store t=2024-04-05T13:25:17.075217023+02:00 level=info msg="Loading plugins..."
logger=plugins.registration t=2024-04-05T13:25:17.158367813+02:00 level=info msg="Plugin registered" pluginId=input
logger=plugins.registration t=2024-04-05T13:25:17.500244057+02:00 level=info msg="Plugin registered" pluginId=grafana-mqtt-datasource
logger=plugin.store t=2024-04-05T13:25:17.520383495+02:00 level=info msg="Plugins loaded" count=57 duration=445.168417ms
logger=query_data t=2024-04-05T13:25:17.536292868+02:00 level=info msg="Query Service initialization"
logger=live.push_http t=2024-04-05T13:25:17.544874591+02:00 level=info msg="Live Push Gateway initialization"
logger=ngalert.migration t=2024-04-05T13:25:17.563468868+02:00 level=info msg=Starting
logger=ngalert.state.manager t=2024-04-05T13:25:17.615028724+02:00 level=info msg="Running in alternative execution of Error/NoData mode"
logger=infra.usagestats.collector t=2024-04-05T13:25:17.620206109+02:00 level=info msg="registering usage stat providers" usageStatsProvidersLen=2
logger=server t=2024-04-05T13:25:17.620490587+02:00 level=info msg="Writing PID file" path=/run/grafana/grafana-server.pid pid=271408
logger=provisioning.alerting t=2024-04-05T13:25:17.622387021+02:00 level=info msg="starting to provision alerting"
logger=provisioning.alerting t=2024-04-05T13:25:17.622474724+02:00 level=info msg="finished to provision alerting"
logger=ngalert.state.manager t=2024-04-05T13:25:17.623406543+02:00 level=info msg="Warming state cache for startup"
logger=ngalert.multiorg.alertmanager t=2024-04-05T13:25:17.62424329+02:00 level=info msg="Starting MultiOrg Alertmanager"
logger=ngalert.state.manager t=2024-04-05T13:25:17.625047241+02:00 level=info msg="State cache has been initialized" states=0 duration=1.635439ms
logger=ngalert.scheduler t=2024-04-05T13:25:17.625299922+02:00 level=info msg="Starting scheduler" tickInterval=10s maxAttempts=1
logger=ticker t=2024-04-05T13:25:17.625475753+02:00 level=info msg=starting first_tick=2024-04-05T13:25:20+02:00
logger=http.server t=2024-04-05T13:25:17.628271859+02:00 level=info msg="HTTP Server Listen" address=[::]:3000 protocol=http subUrl= socket=
logger=grafanaStorageLogger t=2024-04-05T13:25:17.637164688+02:00 level=info msg="Storage starting"
logger=provisioning.dashboard t=2024-04-05T13:25:17.721609904+02:00 level=info msg="starting to provision dashboards"
logger=provisioning.dashboard t=2024-04-05T13:25:17.721719032+02:00 level=info msg="finished to provision dashboards"
logger=grafana.update.checker t=2024-04-05T13:25:17.791794703+02:00 level=info msg="Update check succeeded" duration=163.123406ms
logger=plugins.update.checker t=2024-04-05T13:25:17.798990484+02:00 level=info msg="Update check succeeded" duration=169.039021ms
logger=grafana-apiserver t=2024-04-05T13:25:18.757497222+02:00 level=info msg="Adding GroupVersion playlist.grafana.app v0alpha1 to ResourceManager"
logger=grafana-apiserver t=2024-04-05T13:25:18.759388045+02:00 level=info msg="Adding GroupVersion featuretoggle.grafana.app v0alpha1 to ResourceManager"
logger=context userId=1 orgId=1 uname=admin t=2024-04-05T13:25:27.426763729+02:00 level=info msg="Request Completed" method=GET path=/api/live/ws status=-1 remote_addr=192.168.178.20 time_ms=5 duration=5.987911ms size=0 referer= handler=/api/live/ws status_source=server
logger=live t=2024-04-05T13:25:29.382934204+02:00 level=info msg="Initialized channel handler" channel=grafana/dashboard/uid/ddhk14q175k3kd address=grafana/dashboard/uid/ddhk14q175k3kd
logger=plugin.grafana-mqtt-datasource t=2024-04-05T13:25:29.540121367+02:00 level=info msg="MQTT Connecting"
logger=live t=2024-04-05T13:25:29.574998603+02:00 level=info msg="Initialized channel handler" channel=ds/adhk0hz66yigwd/30s/plant_humidity_1 address=ds/adhk0hz66yigwd/30s/plant_humidity_1
logger=infra.usagestats t=2024-04-05T13:26:00.666490603+02:00 level=info msg="Usage stats are ready to report"
logger=live t=2024-04-05T13:26:43.764124377+02:00 level=info msg="Initialized channel handler" channel=ds/adhk0hz66yigwd/500ms/plant_humidity_1 address=ds/adhk0hz66yigwd/500ms/plant_humidity_1
logger=live t=2024-04-05T13:27:58.814757712+02:00 level=info msg="Initialized channel handler" channel=ds/adhk0hz66yigwd/1s/plant_humidity_1 address=ds/adhk0hz66yigwd/1s/plant_humidity_1
logger=cleanup t=2024-04-05T13:35:17.776219972+02:00 level=info msg="Completed cleanup jobs" duration=151.846518ms
logger=plugins.update.checker t=2024-04-05T13:35:17.950292236+02:00 level=info msg="Update check succeeded" duration=150.883994ms
logger=live t=2024-04-05T13:35:50.900779495+02:00 level=info msg="Initialized channel handler" channel=ds/adhk0hz66yigwd/2s/plant_humidity_1 address=ds/adhk0hz66yigwd/2s/plant_humidity_1
logger=cleanup t=2024-04-05T13:45:17.781328937+02:00 level=info msg="Completed cleanup jobs" duration=157.031952ms
logger=plugins.update.checker t=2024-04-05T13:45:17.932664325+02:00 level=info msg="Update check succeeded" duration=133.118933ms

[michaelw]

Looks like this is actually the default behaviour and not related MQTT at all, see link. Quite surprised about that though.

[yosiasz]

@michaelw

Seems like the data is not stored properly.

Data stored where?

[michaelw]

I was assuming it is stored in the default sqlite DB. But it seems like it is not, this DB is just used for UI settings, dashboards etc.
However, I wonder how to use the MQTT plugin reasonably then? If I just see the data received since I opened the dashboard and no historic data it doesn’t make sense.

[yosiasz]

Depends. Mqqt retention etc maybe? So if you ran the same query outside of grafana through some querying tool of mqqt, would you see historical data?

[michaelw]

First thanks for trying to help me with this. Didn’t notice the reply though.

Unfortunately this is not how MQTT works. I use a so called “MQTT broker” (Mosquitto) and I think the methodology is similar to Apache Kafka. You can publish information via “topics” to send information. To receive information the receiving program has to subscribe to that specific topic. Only after subscription it will receive relevant data and any data will only be received once.

So that’s why I wonder how a proper Grafana MQTT dashboard should be configured? Of course I could write a little program to collect the MQTT data and write it to a database. Then I could query this DB from Grafana. But what is the point in the MQTT data source then? How am I supposed to use it properly when data is stored nowhere?

[yosiasz]

@michaelw

But what is the point in the MQTT data source then? How am I supposed to use it properly when data is stored nowhere?

well why did you decide to use mqqt then?

see this new thread in case it helps

MQTT chart demo

This is a short but detailed example of how to get MQTT data into GRAFANA using the MQTT data source. Whilst there are several examples they all lack detail so I thought I would cover this. I have an electrical meter with MQTT feature enabled. I used the HIVEMQ Public Broker tcp://broker.hivemq.com:1883 and it connected no problem. It requires no user credentials, SSL/TLS or Last Will and Testament (LWT). However HIVEMQ requires one to login with username and password and it is these details wh…

[expoenergy]

Hi @michaelw,

I think the subtle difference is that once a dashboard is saved GRAFANA streams data directly to the panel data buffer which you may readily access and download etc.

However as soon as any dashboard change is made and saved the panel data buffer/queue is purged as you note.

Below is an MQTT dashboard which has just started steaming data as you can see

Then I edit the panel and as you can well see when in edit mode the data panel remains static.

If I now discard any changes the streaming is unbroken and all data is retained.

This is ideally how you wish matters would work whether one edits or discards.

Below is what happens when I make any change which is undesireable as you note

Out of interest the exact same issues arise with the node-red dashboard chart whether it be version 1 or version 2 dashboard charts.

Node-red has a similar issues so it is not unique to GRAFANA.

Maybe if the GRAFNA panel could distinguish between cosmetic dashboard changes opposed to query changes then it might do what you require but that would be a hell of a mission to do.

[michaelw]

Hi @expoenergy,

I could live with the behaviour you describe. But my issue is not even related to changing the dashboard itself. I just navigate to other areas within Grafana and anytime I open the dashboard it is empty again - just collecting the data from the time I opened it and then being initialized whenever I leave it and come back.

[grant2]

Just to chime in here, when I first set up MQTT with Grafana, I too was disappointed that just closing the dashboard and then coming back would cause the 5 minutes or whatever of history to disappear and then start again. For that reason, today the only panel that I use the MQTT plugin with his the Stat Panel, which essentially gives me a nearly instantaneous value of the parameter that I am monitoring.