How to parse esxi hosts logs?

Grafana Loki
1

Hi there,

I’m new to Grafana + loki + promtail and try to visualize logs from esxi hosts. I managed to see the logs in Grafana dashboard but would like to optimize the view. I have following settings:

Esxi host syslog setting:

image
image783×343 9.8 KB

Promtail setting:
image

Loki setting:

image
image450×810 12 KB

With the syslog setting in esxi hosts, I expect to receive only “Error” logs, but it still show “Info” logs in the dashboard:

image

  1. May I know is it related to Promtail or Loki setting that it won’t filter the logs?

  2. How can I include the header in the log like the way vmware does?
    e.g.
    image

2

I am not sure what you mean by this, I don’t see any sort of filtering being done in your promtail configuration

Maybe one of the other syslog labels might have the information you need (maybe __syslog_message_proc_id?) See Configure Promtail | Grafana Loki documentation

3

Thanks Tony,

For Q1, I mean I already set the log filter level in Esxi host setting, but all logs still showing in the dashboard, so I guess it does not related to the Promtail or Loki setting but VMware’s problem?

And I found it is confusing about the severity.

image

From the original log it should be “Warning” but it detected as “Error” and the label “__syslog_message_severity” shows notice…

I just quickly googled and found some discussion about this on the internet so I will continue to research, of course any tips would be appreciated :stuck_out_tongue:

For Q2, I managed to get the required field by using the label “__syslog_message_app_name”
image

but it shows only when I expand the log entry, not showing in the dashboard:

image
image698×132 25.1 KB

Is it possible to show this filed in the dashboard?

4

Labels should be kept as labels (meaning not part of the logs, but rather metadata that describes the logs). You can, if you really want to, alter the logs with promtail, but I personally don’t like altering logs during transit.

In promtail you can set any parsed field as output. So you can manipulate a field with template and add two fields together (or a field and the log body), then set the new field as the output.