Elasticsearch data source how to limit fields

Grafana Elasticsearch
1

  • What Grafana version and what operating system are you using?
    Grafana v10.4.2

  • What are you trying to achieve?
    Currently connected to elasticsearch data source. I want to create a table by using the logs but I want to limit the columns shown.

  • How are you trying to achieve it?
    I tried 2 approahes:

  1. Transfrom data (for tables). >> Organize fields by name >>Hide each of the fields except for the ones I want to display
  2. Override, using regex: ^(?!@source_timestamp$|event_severity$|label$).+$ This kinda works so it hides everything except for the values in the parenthesis
  3. Lucene: _fields: ["@source_timestamp","event_severity", "label"]
  • What happened?
  1. works visually, extremely tedious if there are a lot of fields. If the field list grows it wont hide it
  2. works visually, but im guessing its querying the whole table and filtering out. would rather only query for what I need
  3. does not work but I would like to use this because it doesn’t query the whole table just to get the 3 fields I need
  • What did you expect to happen?
    I expected only 3 fields to appear when using Lucene but I simply cannot get lucene to work properly
2

I would try to select is as metric - count with grouping for all selected fields, which needs to be in the result + count column will be hidden in the final table. E.g.:

obrázok
obrázok1542×413 35.1 KB

Of course there are edge cases where it may miss some logs, but you should know if it may happen with your data,

1 Like
3

I’m using the log type because I don’t need to aggregate the fields just need to display it. Is it not possible to limit the query via lucene? Or am I thinking of this in correctly. Very new to Grafana. I could use count but then its aggregating

4

If you think “I must to use log query”, then no. I gave you idea out of box, so your choice - stick with your “must” or try something what someone recommend you to explore.

5

Hi, I think your idea is great so don’t take any offense by it. I guess I need to preface this by saying I’m trying to learn more on how I can use Lucene to do queries for elastic and why the current lucene command doesnt work.

1 Like
6

maybe try infinity datasource plugin instead by calling url:9200

1 Like
7

oh this is cool just reading about it now. Ok maybe this will work thanks!

1 Like