Show mqtt JSON messages in Grafana

Grafana Live
1

Hi,
We have a running setup with a K3S Cluster, Telegraf for metrics pushed to mqtt and Grafana to show these metrics. We don’t need historical data, so we do not have a InfluxDB installed.
Our problem is, that we can see the JSON messages in Grafana, but we have no clue how to parse them and show them in a dashboard.

MQTT message (e.g. memory):

{
    "fields":{
        "active":442454016,
        "available":2941112320,
        "available_percent":71.30904475702818,
        "buffered":46247936,
        "cached":1229934592,
        "commit_limit":2062229504,
        "committed_as":1570803712,
        "dirty":24576,
        "free":1897500672,
        "high_free":0,
        "high_total":0,
        "huge_page_size":2097152,
        "huge_pages_free":0,
        "huge_pages_total":0,
        "inactive":1673703424,
        "low_free":0,
        "low_total":0,
        "mapped":414302208,
        "page_tables":4644864,
        "shared":1703936,
        "slab":72286208,
        "sreclaimable":41578496,
        "sunreclaim":30707712,
        "swap_cached":0,
        "swap_free":0,
        "swap_total":0,
        "total":4124459008,
        "used":950775808,
        "used_percent":23.052133774534536,
        "vmalloc_chunk":0,
        "vmalloc_total":35184372087808,
        "vmalloc_used":12550144,
        "write_back":0,
        "write_back_tmp":0
    },
    "name":"mem",
    "tags":{
        "host":"vo-node-1"
    },
    "timestamp":1679647780
}

Screenshot from 2023-03-24 09-55-20
Screenshot from 2023-03-24 09-55-201190×354 25.8 KB

As you can see, Grafana gets all the information from the message and if we use this query as a base for a Time Series Dashboard, it will only show the value of “timestamp”.
But we want to show the value for “available_percent” for example and in the next step, show this percentage value for all of our four K3S nodes.
Any help is much appreciated :slight_smile:
Markus

2

There is probably a Telegraf method to parse the MQTT message and separate out what you want, but I would personally do this in NodeRED. Have you considered that option?

3

Hi @grant2

thanks for your reply.
No, not at the moment.
We wanted the cluster as small as possible due to resource limitations.
But I’ll check the Telegraf output.
Thanks
Markus

4

@deschmma
Markus: NodeRED is dead easy to get going and for what you want to do, it would be a simple flow like this:

image
image1163×261 19.7 KB

I do not have the MQTT datasource plugin installed right now (it’s running on another system though, but I won’t have access until tonight). Basically I would set it up to subscribe to the topic “available_percent”.

5

Hi @grant2

what settings do you have in “set msg.payload” and the outgoing mqtt “available_percent”?
Never worked with node-red before :slight_smile:
Thanks
Markus

6

So in my example, I was sending the entire JSON array (just as you posted). That is what shows up in the first Debug node.

image
image943×715 41.2 KB

The object that you want (in your example) is available_percent, which is msg.payload.fields.available_percent, so I put a Change node in there and set the msg.payload to be msg.payload.fields.available_percent:

image
image1134×375 29.9 KB

Then it will pass that one object to the MQTT broker (circled in black). You could send other objects as well and regulate the frequency of how often to check or resend the value:

image
image1059×261 21.8 KB

Here is what the MQTT node settings (the purple one) look like. Basically just enter the IP address of your MQTT broker and set the topic that you want to use:
image

7

ok, that works.
Changed “Inject” to “mqtt in” and it works as well.
I’ll try to grab this new message and import that to the Grafana dashboard.
If it works, then I’ll setup the node-red parse for all values that I need.
Thanks for your support :wink:
Markus

8

Great. Post back or direct message me if you get stuck.