Mixed ElasticSearch datasources

#1

Hi,

I have two indexes. I’m using logstash to index the log files to ElasticSearch.
Index 1
{
“_index” : “my-index”,
“_type” : “doc”,
“_id” : “va_TbWoBfjxVI5P0WKhc”,
“_score” : 1.0,
“_source” : {
@timestamp” : “2019-04-30T10:38:46.065Z”,
“status” : “STARTED”,
@version” : “1”,
“job_name” : “my-job-1”
}
{
“_index” : “my-index”,
“_type” : “doc”,
“_id” : “va_TbWoBfjxVI6P8WKhc”,
“_score” : 1.0,
“_source” : {
@timestamp” : “2019-04-30T10:38:46.065Z”,
“status” : “Completed”,
@version” : “1”,
“job_name” : “my-job-2”
}

Here is the second Index
Index 2

{
        "_index" : "my-index-2",
        "_type" : "doc",
        "_id" : "va_TbWoBfjxVI6P8WKhc",
        "_score" : 1.0,
        "_source" : {
          "@timestamp" : "2019-04-30T10:38:46.065Z",
          "timetaken" : 500,
          "@version" : "1",
          "job_name" : "my-job-2"
        }
		{
        "_index" : "my-index-2",
        "_type" : "doc",
        "_id" : "va_TbWoBfjxVI6P8WKhc",
        "_score" : 1.0,
        "_source" : {
          "@timestamp" : "2019-04-30T10:38:46.065Z",
          "timetaken" : 200,
          "@version" : "1",
          "job_name" : "my-job-1"
        }
		{
        "_index" : "my-index-2",
        "_type" : "doc",
        "_id" : "va_TbWoBfjxVI6P8WKhc",
        "_score" : 1.0,
        "_source" : {
          "@timestamp" : "2019-04-30T10:38:46.065Z",
          "timetaken" : 900,
          "@version" : "1",
          "job_name" : "my-job-2"
        }

I want to display table in Grafana by joining these two indexes.

  1. Job Name
  2. Current Status
  3. Avg timetaken

I have tried the Table visualization and used RawDocument and Json data with column names as job_name & status to display from 1st index in a table. It’s working fine.

Now I want to use Mixed datasource and calculate the average of timetaken from index 2 by grouping job name and display the avg in the same table along with index 1 data by joining job_name.

Please assist on getting my index 2 to work along with index 1.

#2

You can’t join data using the mixed datasource to do aggregations of data. Would suggest you create an index alias in elasticsearch that includes both of your indices and then in grafana you configure one elasticsearch datasource to target this index alias as index pattern.