Grafana cannot display some text fields


#1

Hi,

I’m using the http_poller to pull some data from a site. The data arrives in JSON format and I have my filters, etc set up and all works as expected in the ELK stack. Looking in Kibaba I can see the data as expected.
However, when I try to render the same data in Grafana, I get mixed results as to whether it shows a text field. The layout of the index is shown below. The only fields with issues is the “summary” field.

{
  "cve-last-30-2019.03.14": {
    "aliases": {},
    "mappings": {
      "doc": {
        "properties": {
          "@timestamp": {
            "type": "date"
          },
          "@version": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "Modified": {
            "type": "date"
          },
          "Published": {
            "type": "date"
          },
          "cvss": {
            "type": "float"
          },
          "cwe": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "id": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "last-modified": {
            "type": "date"
          },
          "nist_link": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "ranking": {
            "properties": {
              "circl": {
                "type": "long"
              }
            }
          },
          "references": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "summary": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "tags": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "vulnerable_configuration": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "vulnerable_configuration_cpe_2_2": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "watch": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      }
    },
    "settings": {
      "index": {
        "refresh_interval": "5s",
        "number_of_shards": "1",
        "provided_name": "cve-last-30-2019.03.14",
        "creation_date": "1552577501618",
        "number_of_replicas": "2",
        "uuid": "Uq_HRhBtSQesxIM0zyA2Zg",
        "version": {
          "created": "6020199"
        }
      }
    }
  }
}

Adding the “Missing” tag of “Cannot render Summary” in Grafana for the summary field to ensure it shows up to troubleshoot will result in this:


Take entry “CVE-2019-9752” as an example, you can see that the summary field isn’t generated for this entry. Searching for that same entry in Kibana produces the following:

If I wasn’t getting anything shown for all summary fields in Grafana, I would put it down to how the index was set up but that’s not the case so I’m at a loss to figure out why I don’t see anything.

Is there anything obvious I should be checking?


#2

Check query inspector - any special characters saved in the summary field? Check also browser console. Did you configure column style?


#3

Thanks for the reply. The column style is configured but only as a string.
A sample of a summary that isn’t displayed:

FeiFeiCMS 4.1.190209 allows remote attackers to upload and execute arbitrary PHP code by visiting index.php?s=Admin-Index to modify the set of allowable file extensions, as demonstrated by adding php to the default jpg,gif,png,jpeg setting, and then using the “add article” feature.

Would “?”, “=” or quotes in the field really cause an issue? I assumed as it’s just a string field, Grafana would just ignore it and output it “as is”. Is this not the case? What do I need to change to make this work as expected?!

Thanks.


#4

Been working on this this afternoon and it’s still escaping me what the issue is.

This entry works:

   {
      "_index": "cve-last-30-2019.03.15",
      "_type": "doc",
      "_id": "gwJogmkBUwhlFVZvVfPC",
      "_version": 1,
      "_score": null,
      "_source": {
        "references": [
          "https://improsec.com/tech-blog/cam1"
        ],
        "@timestamp": "2019-03-15T17:31:09.767Z",
        "tags": [
          "cvelast30"
        ],
        "alert": "0",
        "Modified": "2019-03-15T11:29:00.583000",
        "cvss": null,
        "last-modified": "2019-03-15T11:29:00.583000",
        "summary": "An issue was discovered in CapMon Access Manager 5.4.1.1005. A regular user can obtain local administrator privileges if they run any whitelisted application through the Custom App Launcher.",
        "nist_link": "https://nvd.nist.gov/vuln/detail/CVE-2018-18256",
        "@version": "1",
        "cwe": "Unknown",
        "id": "CVE-2018-18256",
        "Published": "2019-03-15T11:29:00.567000"
      },
      "fields": {
        "last-modified": [
          "2019-03-15T11:29:00.583Z"
        ],
        "@timestamp": [
          "2019-03-15T17:31:09.767Z"
        ],
        "Modified": [
          "2019-03-15T11:29:00.583Z"
        ],
        "Published": [
          "2019-03-15T11:29:00.567Z"
        ]
      },
      "highlight": {
        "id": [
          "@kibana-highlighted-field@CVE@/kibana-highlighted-field@-@kibana-highlighted-field@2018@/kibana-highlighted-field@-@kibana-highlighted-field@18256@/kibana-highlighted-field@"
        ]
      },
      "sort": [
        1552671069767
      ]
    }

Where as this one does not:

{
  "_index": "cve-last-30-2019.03.15",
  "_type": "doc",
  "_id": "ewJogmkBUwhlFVZvVfPC",
  "_version": 1,
  "_score": null,
  "_source": {
    "references": [
      "https://improsec.com/tech-blog/cam1"
    ],
    "@timestamp": "2019-03-15T17:31:09.767Z",
    "tags": [
      "cvelast30"
    ],
    "alert": "0",
    "Modified": "2019-03-15T11:29:00.537000",
    "cvss": null,
    "last-modified": "2019-03-15T11:29:00.537000",
    "summary": "An issue was discovered in CapMon Access Manager 5.4.1.1005. The client applications of AccessManagerCoreService.exe communicate with this server through named pipes. A user can initiate communication with the server by creating a named pipe and sending commands to achieve elevated privileges.",
    "nist_link": "https://nvd.nist.gov/vuln/detail/CVE-2018-18255",
    "@version": "1",
    "cwe": "Unknown",
    "id": "CVE-2018-18255",
    "Published": "2019-03-15T11:29:00.520000"
  },
  "fields": {
    "last-modified": [
      "2019-03-15T11:29:00.537Z"
    ],
    "@timestamp": [
      "2019-03-15T17:31:09.767Z"
    ],
    "Modified": [
      "2019-03-15T11:29:00.537Z"
    ],
    "Published": [
      "2019-03-15T11:29:00.520Z"
    ]
  },
  "highlight": {
    "id": [
      "@kibana-highlighted-field@CVE@/kibana-highlighted-field@-@kibana-highlighted-field@2018@/kibana-highlighted-field@-@kibana-highlighted-field@18255@/kibana-highlighted-field@"
    ]
  },
  "sort": [
    1552671069767
  ]
}

There is nothing in the summary field that is special that would indicate why Grafana can display one over the other…