I am a little struggling to understand how to deploy Grafana Live streaming introduced in v8. I would like to use a Raspberry Pi, that reads (via Python) some sensors over I2C and streams the data to Grafana. For the live update, I would like to acquire the sensor data every ~200-500 ms. Furthermore, I would like to also store long-term data in a InfluxDB e.g. the average value every 10s.
So far (without the live streaming), we typically ran the data acquisition within Python and used the influxdb client in Python to push the data to the server. We had setup a systemd timer to execute the Python script every 10 seconds.
What would now be the easiest way to setup live streaming? Maybe shift over to telegraf (then using the execd input plugin + modified Python scripts)? But how do I then get the long-term storage in InfluxDB? Do I stream only to Grafana and Grafana forwards to InfluxDB? How do we merge live data & Influx data in the panel of Grafana?
We have the exact same need as you described and nearly the exact same setup, except we generally push data to Influx using Node-RED (not Telegraf). Regardless, the live streaming in Grafana you have heard about it still in beta. If you know how to do it, you can try to build the MQTT datasource plug in. I tried but never could get it. I believe we are now just a few weeks away from having the ‘official’ MQTT datasource plugin available, so if you can wait, I think this will become very simple.
Are you familiar with MQTT? It’s pretty simple to set up. Once you set up an MQTT broker, it will serve as a sort of ‘post office’ to for the various clients to publish and subscribe to. Then Grafana will be configured with the MQTT datasource with the IP address of the MQTT broker.
PS: Forgot to answer your original question…You would use Telegraf to send data to both Influx and the MQTT broker. That way you can store the data for historical viewing and stream the data (from the MQTT broker) for live streaming.
thanks for you reply! Yes, I am a little familiar with MQTT. I see now that then MQTT would become the distribution hub for the sensor values. So far I have used the Telegraf MQTT input plugin to forward MQTT traffic to an InfluxDB. However, my sensor will now stream the values e.g. every 200 ms to the MQTT broker. Then all of them will also endup in the InfluxDB. How can I now reduce the amount of data points stored in Influx (e.g. to 1 data point every 10s or so)?
I have a similar case but I use C# InfluxDB client to send data to InfluxDB. And I would like show in live way the data in Grafana. If I understand I need to use MQTT and telegraph ?
@patopat Welcome to the Grafana forum. Just install the MQTT datasource plugin and then Grafana can read the data live. No Telegraf or Influx install needed.
