I can't display my elasticsearch index in table panel

Grafana Elasticsearch
1

Hi everyone,

Currently i’m sending a log file (json format) from FileBeat (Windows Server) to Logstash (parsing json file) then from logstash to elasticsearch then I want to retrieve this data in Grafana with a table panel.

Source Log :

{“@timestamp”:“2018-08-13T23:07:22.905+00:00”,“@version”:“1”,“message”:“Started ApplicationWebXml in 43.358 seconds (JVM running for 233.488)”,“logger_name”:“com.XXX.XXX.XXX.ApplicationWebXml”,“thread_name”:“XXX.XXX.XXX-startStop-1”,“level”:“INFO”,“level_value”:20000}

FileBeat :

filebeat.inputs:

  • type: log
    enabled: true
    paths:
    • C:\LOGS*
      output.logstash:
      hosts: [“192.168.1.113:5044”]

Logstash :

input {
beats {
port => “5044”
}
}
filter {
json {
source => “message”
}
}
output {
elasticsearch {
hosts => [ “192.168.1.112:9200” ]
}
}

Elasticsearch (example from Kibana) :

Elasticsearch - Indicies :

  • logstash-2018.08.13
  • logstash-2018.08.14

I can see my json message is correctly parsed and stored in elasticsearch :

JSONParsing_Kibanaview
JSONParsing_Kibanaview1116×718 33.1 KB

Grafana - DataSource :

My data source is correctly configured ( I can’t post a screenshot due to limit to 2 for new users).

Time field is @timestamp

Grafana - Table Panel :

TablePanel_withoutESDATA
TablePanel_withoutESDATA1840×662 82.6 KB

I select Elasticsearch source but I can not retrieve correctly my ES data …

I just want to display fields we can see in kibana screenshot into grafana table panel.

I’m using Elasticsearch 6.4.1.

Thanks a lot, each help will be warmly welcomed :slight_smile:

2

I guess you want to see RAW documents in the table, without any aggregation. Select Metric: RAW Document:

image
image996×219 5.61 KB

In Options section Table Transform: JSON Data + select columns, which you want to show in the table:

image
image1024×485 17.5 KB

Doc: http://docs.grafana.org/reference/table_panel/

3

Thanks for your answer,

I select Raw Document :

Raw%20Document
Raw%20Document1833×737 55.7 KB

Options :

Options
Options1847×698 44.9 KB

In columns field I can’t get my properties such as @timestamp message level logger_name etc.

I’ve read the documentation but i don’t get it why I can’t select these properties in columns field :s

4

Because of columns fields in UI are generated from the received documents - probably you don’t have any docs in ES for last hour or @timestamp is not in UTC. Try to select last 30 days instead of last hour.

You may also try - https://grafana.com/dashboards/635

5

What worked for me:

As mentioned above, first choose Raw Document in Metric.
You need to make sure you do have documents matching criteria (time series selected).
Once you’ve done that (and that was my issue), click on the “Refresh dashboard” link in the top right corner.

What does not work for me, is when I narrow down the fields, I still get ALL of them displayed in the table. Any help is appreciated.