ElasticSearch return latest document

Hi Grafana community,

I’m trying to do something in Grafana and I am not even sure if it’s supported. We are using Grafana version 5.2.4 with ElasticSearch 6.2 and what I’ve been asked to do is to create a table panel with the values for specific fields in our ElasticSearch data. The issue I’m trying to overcome is that I only want the values (which are generally text) for the latest version of the event. I have a “documentId” field and essentually I want to return only the latest event containing that documentId when displaying the data for the table. I think I need to do some kind of term aggregation on documentId, but when I use “Raw Document” for the metric so I can create the table, there is no way to aggregate on the documentId that I can see. The only thing I can set there is the size (number of returned events).

Am I missing something on how to do this? Am I providing enough information?


Let me give a little more info. Say I have this data in elasticsearch:

{ “documentId”: 1, “data”: { “text”:“hello”, “user”:“bob”}, “@timestamp”: “2019-06-11 08:34:24” }
{ “documentId”: 2, “data”: { “text”:“hi”, “user”:“sam”}, “@timestamp”: “2019-06-11 08:35:34” }
{ “documentId”: 3, “data”: { “text”:“greetings”, “user”:“mary”}, “@timestamp”: “2019-06-11 08:35:37” }
{ “documentId”: 1, “data”: { “text”:“hola”, “user”:“bob”}, “@timestamp”: “2019-06-11 08:37:54” }
{ “documentId”: 4, “data”: { “text”:“hello”, “user”:“joan”}, “@timestamp”: “2019-06-11 08:38:01” }
{ “documentId”: 2, “data”: { “text”:“howdy”, “user”:“sam”}, “@timestamp”: “2019-06-11 08:38:11” }

I would want a table with only the most recent data from each document, so it would show document 2 for instance, but it would have only the data from the last line in the example since it’s newer.

Is that even possible with “Raw Document” as the metric?