Elastic Raw Data Transform for Status History Panel

Version: Grafana v10.4.2

I have a dataset from an Elastic index that I have condensed down to timestamp, host, and status columns using “Organize fields by name”. I wish to use this data to generate a status history for each of the hosts. Timestamps are not unique because for each timestamp sample there are 500+ hosts and a status for each host. The hope is to be able to expand on this at a later time, adding more “status” information in the future or just generating a similar but expanded plot.

Incoming data looks like this (in Table View):

@timestamp hostname status
2024-09-21 12:00:03 host12 up
2024-09-21 12:00:03 host11 down
2024-09-21 11:45:03 host12 up
2024-09-21 11:45:03 host11 up

I would like the data to look like this to match the requirements of the Status History visualization:

@timestamp host12 host11
2024-09-21 12:00:03 up down
2024-09-21 11:45:03 up up

I’ve tried just about every transform possible but haven’t come up with the correct incantations and pinches of salt over the shoulder because “Grouping to matrix” casts the @timestamp value to str and gives me nothing usable. And if there is a better way to do this, I am more than willing to give it a go.

Thank you for your input!

nick

Welcome @nicbarry

Is it expected that there’s only one timestamp? If so, you can hopefully use Convert field type transformation (or something like that) to convert it back to time.

(Sorry, I realized that my previous post, since deleted, had somewhat sensitive info in it)

Alas:

I had tried that Transform but clearly I’m not doing something correctly.

1 Like

The unadulterated data looks like this (once transformed via Organize fields by name):

So for any given timestamp, there will be one entry for each system being polled.

If I’m reading the visualization plugin information correctly, I need the first column to be a unique value for timestamp and then each column header after that will be a hostname with the status as its value.

Why do you think you’re doing something incorrectly? Seems to me (correct me if I’m wrong) that the screen provides you with a result (only missing piece might be adding another transformation Convert field type and changing the type of the column from String to time:

Notice, that you have only one timestamp from the next screen you’ve shared, so grafana created one row (default limit for elasticsearch logs is 500, so maybe you don’t have any more data to begin with?).

And the last “Convert field type” transform was the key. Thank you!

1 Like