MQTT Data in HTML panel

[yendor]

Hi,
I’ve been playing around trying to get some mqtt interaction into a panel and was wondering if there is anyone else looking at doing this as well?

My first pass is to use the Paho javascript client, it kind of works but disconnects and reconnects a lot. (still tracking this down).

I would really like a panel plug-in maybe I’ll give it a go when I get some spare time.

Here my attempt, all suggestions/comments are welcome. I just added to following to a text panel as html. (with my details of my broker)

<h1>MQTT</h1>
<div id="mqtt_panel01">
    <div>Topic: <span id="topic01">#</span><span>, Status: </span><span id="status01">#</span></div>
    <button class="btn btn-inverse" type="button" onclick="publish('on','test/grafana/panel/status', 0)">On</button>
    <button class="btn btn-inverse" type="button" onclick="publish('off','test/grafana/panel/status', 0)">Off</button>
    <div><span id="request01">#</span></div>
</div>

<script>
    var mqtt;
    var reconnectTimeout = 5000;
    var host = 'mqtt.example.com';
    var port = 8883;
    var topic = 'test/grafana/panel/status';
    var useTLS = true;
    var username = "user";
    var password = "user1234";
    var cleansession = true;
    var path = "";
    var buttons = $('[type=button]');

    buttons.prop('disabled', true);

    function MQTTconnect() {
        if (typeof path == "undefined") {
            path = '/mqtt';
        }
        mqtt = new Paho.MQTT.Client(
            host,
            port,
            path,
            "web_grafanaPanel" + parseInt(Math.random() * 10000, 10)
        );
        var options = {
            timeout: 3,
            useSSL: useTLS,
            cleanSession: cleansession,
            onSuccess: onConnect,
            onFailure: function (message) {
                $('#status01').html("Connection failed: " + message.errorMessage + "Retrying");
                setTimeout(MQTTconnect, reconnectTimeout);
            }
        };
        mqtt.onConnectionLost = onConnectionLost;
        mqtt.onMessageArrived = onMessageArrived;
        if (username != null) {
            options.userName = username;
            options.password = password;
        }
        // console.log("Host=" + host + ", port=" + port + ", path=" + path + " TLS = " + useTLS + " username=" + username + " password=" + password);
        mqtt.connect(options);
    }

    function onConnect() {
        $('#status01').html('Connected to ' + host + ':' + port + path);
        // Connection succeeded; subscribe to our topic(s)
        mqtt.subscribe(topic, {
            qos: 0
        });
        $('#topic01').html(topic);
        buttons.prop('disabled', false);
    }

    function onConnectionLost(response) {
        setTimeout(MQTTconnect, reconnectTimeout);
        $('#status01').html("connection lost: " + response.errorMessage + ". Reconnecting");
        buttons.prop('disabled', true);
    };

    function onMessageArrived(message) {
        var topic = message.destinationName;
        var payload = message.payloadString;
        $('#request01').html(payload);
    };

    var publish = function (payload, topic, qos) {
        // console.log('Sending: ' + JSON.stringify(payload));
        var message = new Paho.MQTT.Message(payload);
        message.destinationName = topic;
        message.qos = qos;
        mqtt.send(message);
    }

    if (typeof Paho === 'undefined') {
        var script = document.createElement('script');
        document.body.appendChild(script);
        script.onload = function () {
            MQTTconnect();
        }
        script.src = 'https://cdnjs.cloudflare.com/ajax/libs/paho-mqtt/1.0.2/mqttws31.min.js';
    } else {
       MQTTconnect();
    }
</script>

Regards,

[amotl]

Dear @yendor,

while I find your idea to display live data from MQTT directly in a panel interesting, I would like to point out a more conventional way where measurement data arriving over MQTT will get ingested into a timeseries database (here: InfluxDB) before being accessed by Grafana.

We answered a question coming from a similar context over here at

Simple IoT platform with MQTT

Dear Siavash, you might also want to have look at Kotori, which is just that: A simple (but powerful) platform for ingesting data from MQTT and visualizing it in Grafana. Cheers, Andreas. P.S.: Disclaimer: I’m the main author of Kotori. Feel free to ask me anything, I will be happy to help getting started with it if this sparks your interest.

With kind regards,
Andreas.

[yendor]

Thanks @amotl for your comment,
I agree normally we display data that is in a timeseries database, but I was toying with the idea of using Grafana for control as well.

In this case I would like to send commands via MQTT and see feedback / status. At the moment we use the Node-Red UI for this function but it would by nice to do it all from one interface.

I suppose we could embed Grafana in a Node-Red HTML element or vice versa.

Regards,

[amotl]

Dear @yendor,

@yendor

I was toying with the idea of using Grafana for control as well. In this case I would like to send commands via MQTT and see feedback / status.

Thanks, now I am getting your whole point. I probably should have scanned your code better.

This very much resonates with some thoughts we also already had when discussing the command & control channel to communicate back to upstream nodes within the Hiveeyes Project and how that would be operated exactly on top of the MQTT substrate.

@yendor

At the moment we use the Node-Red UI for this function but it would by nice to do it all from one interface. I suppose we could embed Grafana in a Node-Red HTML element or vice versa.

While Node-RED is really powerful, we haven’t yet got friends as we wouldn’t know how to operate a multi-tenant non-interactive setup with it. As we just recently started to get better friends with Grafana and its internal interfaces in the course of [1-4], we would go for the “native way” of connecting the Browser to MQTT by MQTT over WebSockets, which in the best case would yield a Panel Plugin which can be operated in a standalone manner without depending on Node-RED. However, it still could interact with Node-RED using MQTT like with any other bus member.

In fact, this would be pretty much what you implemented and shared already. Thanks!

@yendor

it kind of works but disconnects and reconnects a lot

The reason for that is probably because the panel gets refreshed by the Grafana machinery on each refresh interval and maybe on additional conditions. So, by just folding all the connection management into a real Grafana plugin should already solve that issue.

Good to hear from people having similar ideas about how to bring MQTT closer to Grafana in a sensible way. Maybe we can find some time to work on a basic implementation which would go into that direction. Would you be able to share a screenshot of what your Node-RED interface for sending MQTT messages and displaying status information currently looks like to get an idea about where we should aim at interface-wise?

With kind regards,
Andreas.

[1] Giving the Grafana Worldmap Panel some love
[2] Ingesting hourly data from a JSON API
[3] Use "grafana-wtf" to search through all entities of a Grafana instance
[4] GitHub - panodata/grafanimate: Animate timeseries data with Grafana.

[yendor]

There not much to the interface just the standard buttons and a SVG, the colour of the svg change to show state currently stop and in manual.

A typical pump control and status

More than happy to help out, but I am very time poor right now (trying to get a Lorawan network setup and stable)

Regards,

[amotl]

@yendor

A typical pump control and status

This looks clean and reasonable, thanks for sharing!

@yendor

very time poor right now (trying to get a Lorawan network setup and stable)

Good luck!

[amotl]

Dear @yendor,

@yendor

I’ve been playing around around trying to get some mqtt interaction into a panel and was wondering if there is anyone else looking at doing this as well?

We found others are looking into the same topic, so we tried to summarize our findings appropriately.

Using Grafana for submitting downlink commands

With kind regards,
Andreas.

[justain]

Hello @amotl,

You can use a text panel in html mode and use the Grafana API to insert or update data to a data source, see Is it possible to execute sql queries from grafana APIs?. As they say this is a not documented feature and can change in the future.

I created a text panel with this test:

<script type="text/javascript" language="javascript">
function UserAction() {
    var xhttp = new XMLHttpRequest();
    xhttp.open("POST", "/api/tsdb/query", true);
    xhttp.setRequestHeader("Content-type", "application/json");
    xhttp.setRequestHeader("Authorization", "Bearer eyJrIjoiWG9RV1lpbXh0a3RHUkJKUHlaVVBON1Z5djVON05pZUoiLCJuIjoidGVzdGtleSIsImlkIjoxfQ==");
    xhttp.send(JSON.stringify({"from":"1598166093992","to":"1598187693992","queries":[{"refId":"A","intervalMs":60000,"maxDataPoints":1150,"datasourceId":1,"rawSql":"insert into table values (1)","format":"time_series"}]}));
    var response = JSON.parse(xhttp.responseText);
}
</script>
<button type="submit" onclick="UserAction()">Clickme</button>

This may be simpler I just copied the json from a wireshark capture. The Authorization header can be generated in the configuration section, API Keys.
Remember you will need to disable html sanitize in the configuration file.

Once the data is in the database it can be forwarded to the MQTT device with a python script and/or database trigger.

Best regards,
Justino.

[amotl]

Hi there,

there are some updates on this topic, so I want to humbly point out MQTT Button plugin here.

With kind regards,
Andreas.