No vSphere metric up in grafana with influxDB and telegraf

bvivi57130 · June 13, 2020, 12:24pm

Hi
I’m trying to use the following dahboard.

I have deployed the following configuration on a single server (Ubuntu Server 20.04 x64)

InfluxDB v1.6.4
Grafana v7.0.0
Telegraf v1.14.4

I’m looking to supervise a vCenter server in this version: vCenter 6.7 16243230 (U3c)

My telegraf config for vmware is :

## List of vCenter URLs to be monitored. These three lines must be uncommented
## and edited for the plugin to work.
  interval = "20s"
  vcenters = [ "https://vcenter.coolcorp.priv/sdk" ]
  username = "svc_supervision@coolcorp.priv"
  password = "my_password"

  insecure_skip_verify = true
  force_discover_on_init = true

  # Exclude all historical metrics
  datastore_metric_exclude = ["*"]
  cluster_metric_exclude = ["*"]
  datacenter_metric_exclude = ["*"]

  collect_concurrency = 5
  discover_concurrency = 5
## Historical instance
[[inputs.vsphere]]
  interval = "300s"
  vcenters = [ "https://vcenter.coolcorp.priv/sdk" ]
  username = "svc_supervision@coolcorp.priv"
  password = "my_password"

  insecure_skip_verify = true
  force_discover_on_init = true

  # Exclude all historical metrics
  datastore_metric_exclude = ["*"]
  cluster_metric_exclude = ["*"]
  datacenter_metric_exclude = ["*"]

  collect_concurrency = 5
  discover_concurrency = 5

Telegraf uses the following output.

[[outputs.influxdb]]
    urls = ["http://127.0.0.1:8086"]
    database = "sup_infludb"
    timeout = "0s"
    username = "user_sup"
    password = "my_password"

I have a lot of data in the Influx database and Grafana connects well to this same database.
Unfortunately the dashboard remains empty except for the ESXi counter and VMs number.

I start with grafana,influxdb and telegraf. I try to understand the origin of my problem. I have the impression that the variables configured in the dashboard don’t come up with anything.

I tried this query:

SHOW TAG VALUES FROM vsphere_host_cpu WITH KEY=vcenter

I have no result. I feel like no tag is worth anything.

Do you have any idea how to help me?

pmhausen · June 13, 2020, 1:15pm

First use the ˋinfluxˋ command line tool and the ˋshow seriesˋ command - that should give you an overview of how your data is structured. There’s an InfluxDB Admin plugin for Grafana that lets you enter influx commands, too, if you prefer that way.

fadjar340 · June 13, 2020, 3:20pm

Try this configuration in telegraf.conf, I can get all the metric that I need and displai it in Grafana.
I use the same dashboard that you mention above

```
# # Read metrics from VMware vCenter
   [[inputs.vsphere]]
   vcenters = [ "https://vcenter/sdk" ]
      username = "monitoring@vsphere.local"
      password = "BlahBlah"


   vm_include = [ "/*/**"] 
   vm_metric_include = [
     "cpu.demand.average",
     "cpu.idle.summation",
     "cpu.latency.average",
     "cpu.readiness.average",
     "cpu.ready.summation",
     "cpu.run.summation",
     "cpu.usagemhz.average",
     "cpu.usage.average",
     "cpu.used.summation",
     "cpu.wait.summation",
     "mem.active.average",
     "mem.granted.average",
     "mem.latency.average",
     "mem.swapin.average",
     "mem.swapinRate.average",
     "mem.swapout.average",
     "mem.swapoutRate.average",
     "mem.usage.average",
     "mem.vmmemctl.average",
     "net.bytesRx.average",
     "net.bytesTx.average",
     "net.droppedRx.summation",
     "net.droppedTx.summation",
     "net.usage.average",
     "power.power.average",
     "virtualDisk.numberReadAveraged.average",
     "virtualDisk.numberWriteAveraged.average",
     "virtualDisk.read.average",
     "virtualDisk.readOIO.latest",
     "virtualDisk.throughput.usage.average",
     "virtualDisk.totalReadLatency.average",
     "virtualDisk.totalWriteLatency.average",
     "virtualDisk.write.average",
     "virtualDisk.writeOIO.latest",
     "sys.uptime.latest",
   ]
   host_include = [ "/*/host/**"] # Inventory path to hosts to collect (by default all are collected)
   host_metric_include = [
     "cpu.coreUtilization.average",
     "cpu.costop.summation",
     "cpu.demand.average",
     "cpu.idle.summation",
     "cpu.latency.average",
     "cpu.readiness.average",
     "cpu.ready.summation",
     "cpu.swapwait.summation",
     "cpu.usage.average",
     "cpu.usagemhz.average",
     "cpu.used.summation",
     "cpu.utilization.average",
     "cpu.wait.summation",
     "disk.deviceReadLatency.average",
     "disk.deviceWriteLatency.average",
     "disk.kernelReadLatency.average",
     "disk.kernelWriteLatency.average",
     "disk.numberReadAveraged.average",
     "disk.numberWriteAveraged.average",
     "disk.read.average",
     "disk.totalReadLatency.average",
     "disk.totalWriteLatency.average",
     "disk.write.average",
     "mem.active.average",
     "mem.latency.average",
     "mem.state.latest",
     "mem.swapin.average",
     "mem.swapinRate.average",
     "mem.swapout.average",
     "mem.swapoutRate.average",
     "mem.totalCapacity.average",
     "mem.usage.average",
     "mem.vmmemctl.average",
     "net.bytesRx.average",
     "net.bytesTx.average",
     "net.droppedRx.summation",
     "net.droppedTx.summation",
     "net.errorsRx.summation",
     "net.errorsTx.summation",
     "net.usage.average",
     "power.power.average",
     "storageAdapter.numberReadAveraged.average",
     "storageAdapter.numberWriteAveraged.average",
     "storageAdapter.totalReadLatency.average",
     "storageAdapter.totalWriteLatency.average",
     "storageAdapter.read.average",
     "storageAdapter.write.average",
     "sys.uptime.latest",
   ]

    cluster_include = [ "/*/host/**"] # Inventory path to clusters to collect (by default all are collected)
    datastore_include = [ "/*/datastore/**"] # Inventory path to datastores to collect (by default all are collected)
    datastore_metric_include = [
     "disk.used.latest",
     "disk.provisioned.latest",
     "disk.capacity.latest",
     "disk.capacity.provisioned.average",
     "disk.capacity.usage.average",
     "datastore.numberReadAveraged.average",
     "datastore.numberWriteAveraged.average",
    ]

    datacenter_include = [ "/*/host/**"] # Inventory path to clusters to collect (by default all are collected)
    max_query_objects = 64
    max_query_metrics = 64
    collect_concurrency = 3
    discover_concurrency = 2
    force_discover_on_init = true

    object_discovery_interval = "30s"
    timeout = "30s"
    use_int_samples = false
   insecure_skip_verify = true

bvivi57130 · June 13, 2020, 6:03pm

Thanks for your help.

I try this command “show series” and I’ve got a lot of content in the database.

 root@infsrv001:/etc/telegraf/telegraf.d# influx
Connected to http://localhost:8086 version 1.6.4
InfluxDB shell version: 1.6.4
> SHOW DATABASES
name: databases
name
----
_internal
telegraf
sup_infludb
> USE sup_infludb
Using database sup_infludb
> SHOW SERIES
key
---
cpu,cpu=cpu-total,host=infsrv001
cpu,cpu=cpu0,host=infsrv001
cpu,cpu=cpu1,host=infsrv001
disk,device=sda2,fstype=ext4,host=infsrv001,mode=rw,path=/
diskio,host=infsrv001,name=loop0
diskio,host=infsrv001,name=loop1
diskio,host=infsrv001,name=loop2
diskio,host=infsrv001,name=loop3
diskio,host=infsrv001,name=loop4
diskio,host=infsrv001,name=loop5
diskio,host=infsrv001,name=loop6
diskio,host=infsrv001,name=loop7
diskio,host=infsrv001,name=sda
diskio,host=infsrv001,name=sda1
diskio,host=infsrv001,name=sda2
kernel,host=infsrv001
mem,host=infsrv001
processes,host=infsrv001
swap,host=infsrv001
system,host=infsrv001
vsphere_cluster_mem,clustername=CLU,dcname=Paris,host=infsrv001,moid=domain-c846,source=CLU,vcenter=vcenter.coolcorp.priv
vsphere_cluster_vmop,clustername=CLU,dcname=Paris,host=infsrv001,moid=domain-c846,source=CLU,vcenter=vcenter.coolcorp.priv
vsphere_datastore_datastore,dcname=Paris,dsname=instance-total,host=infsrv001,lun=instance-total,moid=datastore-1785,source=VL2,vcenter=vcenter.coolcorp.priv
vsphere_datastore_datastore,dcname=Paris,dsname=instance-total,host=infsrv001,lun=instance-total,moid=datastore-1786,source=VL3,vcenter=vcenter.coolcorp.priv
vsphere_datastore_datastore,dcname=Paris,dsname=instance-total,host=infsrv001,lun=instance-total,moid=datastore-1787,source=VL4,vcenter=vcenter.coolcorp.priv
vsphere_datastore_datastore,dcname=Paris,dsname=instance-total,host=infsrv001,lun=instance-total,moid=datastore-1788,source=VL5,vcenter=vcenter.coolcorp.priv
vsphere_datastore_datastore,dcname=Paris,dsname=instance-total,host=infsrv001,lun=instance-total,moid=datastore-1789,source=VL1,vcenter=vcenter.coolcorp.priv
vsphere_datastore_datastore,dcname=Paris,dsname=instance-total,host=infsrv001,lun=instance-total,moid=datastore-1820,source=MSATA,vcenter=vcenter.coolcorp.priv
vsphere_datastore_datastore,dcname=Paris,dsname=instance-total,host=infsrv001,lun=instance-total,moid=datastore-1821,source=USBDTS,vcenter=vcenter.coolcorp.priv
vsphere_datastore_disk,dcname=Paris,disk=instance-total,dsname=BACKUP,host=infsrv001,moid=datastore-1791,source=BACKUP,vcenter=vcenter.coolcorp.priv
vsphere_datastore_disk,dcname=Paris,disk=instance-total,dsname=MSATA,host=infsrv001,moid=datastore-1820,source=MSATA,vcenter=vcenter.coolcorp.priv
vsphere_datastore_disk,dcname=Paris,disk=instance-total,dsname=NFSCONTAINER,host=infsrv001,moid=datastore-1790,source=NFSCONTAINER,vcenter=vcenter.coolcorp.priv
vsphere_datastore_disk,dcname=Paris,disk=instance-total,dsname=NFSDATASTORE,host=infsrv001,moid=datastore-1784,source=NFSDATASTORE,vcenter=vcenter.coolcorp.priv
vsphere_datastore_disk,dcname=Paris,disk=instance-total,dsname=USBDTS,host=infsrv001,moid=datastore-1821,source=USBDTS,vcenter=vcenter.coolcorp.priv
vsphere_datastore_disk,dcname=Paris,disk=instance-total,dsname=VL1,host=infsrv001,moid=datastore-1789,source=VL1,vcenter=vcenter.coolcorp.priv
vsphere_datastore_disk,dcname=Paris,disk=instance-total,dsname=VL2,host=infsrv001,moid=datastore-1785,source=VL2,vcenter=vcenter.coolcorp.priv
vsphere_datastore_disk,dcname=Paris,disk=instance-total,dsname=VL3,host=infsrv001,moid=datastore-1786,source=VL3,vcenter=vcenter.coolcorp.priv
vsphere_datastore_disk,dcname=Paris,disk=instance-total,dsname=VL4,host=infsrv001,moid=datastore-1787,source=VL4,vcenter=vcenter.coolcorp.priv
vsphere_datastore_disk,dcname=Paris,disk=instance-total,dsname=VL5,host=infsrv001,moid=datastore-1788,source=VL5,vcenter=vcenter.coolcorp.priv
vsphere_host_cpu,clustername=CLU,cpu=0,dcname=Paris,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_cpu,clustername=CLU,cpu=0,dcname=Paris,esxhostname=nucesxi.coolcorp.priv,host=infsrv001,moid=host-1819,source=nucesxi.coolcorp.priv,vcenter=vcenter.coolcorp.priv
vsphere_host_cpu,clustername=CLU,cpu=1,dcname=Paris,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_cpu,clustername=CLU,cpu=1,dcname=Paris,esxhostname=nucesxi.coolcorp.priv,host=infsrv001,moid=host-1819,source=nucesxi.coolcorp.priv,vcenter=vcenter.coolcorp.priv
vsphere_host_cpu,clustername=CLU,cpu=10,dcname=Paris,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_cpu,clustername=CLU,cpu=11,dcname=Paris,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_cpu,clustername=CLU,cpu=12,dcname=Paris,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_cpu,clustername=CLU,cpu=13,dcname=Paris,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_cpu,clustername=CLU,cpu=14,dcname=Paris,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_cpu,clustername=CLU,cpu=15,dcname=Paris,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_cpu,clustername=CLU,cpu=2,dcname=Paris,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_cpu,clustername=CLU,cpu=2,dcname=Paris,esxhostname=nucesxi.coolcorp.priv,host=infsrv001,moid=host-1819,source=nucesxi.coolcorp.priv,vcenter=vcenter.coolcorp.priv
vsphere_host_cpu,clustername=CLU,cpu=3,dcname=Paris,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_cpu,clustername=CLU,cpu=3,dcname=Paris,esxhostname=nucesxi.coolcorp.priv,host=infsrv001,moid=host-1819,source=nucesxi.coolcorp.priv,vcenter=vcenter.coolcorp.priv
vsphere_host_cpu,clustername=CLU,cpu=4,dcname=Paris,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_cpu,clustername=CLU,cpu=5,dcname=Paris,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_cpu,clustername=CLU,cpu=6,dcname=Paris,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_cpu,clustername=CLU,cpu=7,dcname=Paris,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_cpu,clustername=CLU,cpu=8,dcname=Paris,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_cpu,clustername=CLU,cpu=9,dcname=Paris,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_cpu,clustername=CLU,cpu=instance-total,dcname=Paris,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_cpu,clustername=CLU,cpu=instance-total,dcname=Paris,esxhostname=nucesxi.coolcorp.priv,host=infsrv001,moid=host-1819,source=nucesxi.coolcorp.priv,vcenter=vcenter.coolcorp.priv
vsphere_host_datastore,clustername=CLU,dcname=Paris,dsname=BACKUP,esxhostname=esxi,host=infsrv001,lun=d5aa023a-99b15006,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_datastore,clustername=CLU,dcname=Paris,dsname=MSATA,esxhostname=nucesxi.coolcorp.priv,host=infsrv001,lun=5e41e2a7-ac84d1ba-5156-00e04c6858b9,moid=host-1819,source=nucesxi.coolcorp.priv,vcenter=vcenter.coolcorp.priv
vsphere_host_datastore,clustername=CLU,dcname=Paris,dsname=NFSCONTAINER,esxhostname=esxi,host=infsrv001,lun=7eb68f79-f5b9ee55,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_datastore,clustername=CLU,dcname=Paris,dsname=NFSDATASTORE,esxhostname=esxi,host=infsrv001,lun=46f3ea99-41b8a0f9,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_datastore,clustername=CLU,dcname=Paris,dsname=NFSDATASTORE,esxhostname=nucesxi.coolcorp.priv,host=infsrv001,lun=46f3ea99-41b8a0f9,moid=host-1819,source=nucesxi.coolcorp.priv,vcenter=vcenter.coolcorp.priv
vsphere_host_datastore,clustername=CLU,dcname=Paris,dsname=VL1,esxhostname=esxi,host=infsrv001,lun=5ed4b9a6-038f95c7-a90b-001b219adf10,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_datastore,clustername=CLU,dcname=Paris,dsname=VL2,esxhostname=esxi,host=infsrv001,lun=52d13a49-0d3dd5b2-1da7-001b21068f33,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_datastore,clustername=CLU,dcname=Paris,dsname=VL3,esxhostname=esxi,host=infsrv001,lun=574f3997-dc7a36c0-eb6b-fcaa14f04e54,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_datastore,clustername=CLU,dcname=Paris,dsname=VL4,esxhostname=esxi,host=infsrv001,lun=5e179f43-516fa228-daad-001b219adf10,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_datastore,clustername=CLU,dcname=Paris,dsname=VL5,esxhostname=esxi,host=infsrv001,lun=5e179fa4-b8a6bbc6-80e3-001b219adf10,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_datastore,clustername=CLU,dcname=Paris,dsname=instance-total,esxhostname=esxi,host=infsrv001,lun=instance-total,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_datastore,clustername=CLU,dcname=Paris,dsname=instance-total,esxhostname=nucesxi.coolcorp.priv,host=infsrv001,lun=instance-total,moid=host-1819,source=nucesxi.coolcorp.priv,vcenter=vcenter.coolcorp.priv
vsphere_host_disk,clustername=CLU,dcname=Paris,disk=instance-total,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_disk,clustername=CLU,dcname=Paris,disk=instance-total,esxhostname=nucesxi.coolcorp.priv,host=infsrv001,moid=host-1819,source=nucesxi.coolcorp.priv,vcenter=vcenter.coolcorp.priv
vsphere_host_disk,clustername=CLU,dcname=Paris,disk=mpx.vmhba32:C0:T0:L0,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_disk,clustername=CLU,dcname=Paris,disk=mpx.vmhba32:C0:T0:L0,esxhostname=nucesxi.coolcorp.priv,host=infsrv001,moid=host-1819,source=nucesxi.coolcorp.priv,vcenter=vcenter.coolcorp.priv
vsphere_host_disk,clustername=CLU,dcname=Paris,disk=mpx.vmhba33:C0:T0:L0,esxhostname=nucesxi.coolcorp.priv,host=infsrv001,moid=host-1819,source=nucesxi.coolcorp.priv,vcenter=vcenter.coolcorp.priv
vsphere_host_disk,clustername=CLU,dcname=Paris,disk=mpx.vmhba34:C0:T0:L0,esxhostname=nucesxi.coolcorp.priv,host=infsrv001,moid=host-1819,source=nucesxi.coolcorp.priv,vcenter=vcenter.coolcorp.priv
vsphere_host_disk,clustername=CLU,dcname=Paris,disk=t10.ATA_____Crucial_CT120M500SSD3___________________________13410952405C,esxhostname=nucesxi.coolcorp.priv,host=infsrv001,moid=host-1819,source=nucesxi.coolcorp.priv,vcenter=vcenter.coolcorp.priv
vsphere_host_disk,clustername=CLU,dcname=Paris,disk=t10.ATA_____INTEL_SSDSC2BB300G4_____________________CVWL419300DS300PGN__,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_disk,clustername=CLU,dcname=Paris,disk=t10.ATA_____Samsung_SSD_850_PRO_256GB_______________S251NXAH240618D_____,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_disk,clustername=CLU,dcname=Paris,disk=t10.ATA_____Samsung_SSD_860_EVO_500GB_______________S4CNNE0M809644T_____,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_disk,clustername=CLU,dcname=Paris,disk=t10.ATA_____SanDisk_Ultra_II_480GB__________________162224802946________,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.priv
vsphere_host_disk,clustername=CLU,dcname=Paris,disk=t10.ATA_____WDC_WD6000HLHX2D01JJPV0_______________________WD2DWX91EC1CNXJ4,esxhostname=esxi,host=infsrv001,moid=host-1783,source=esxi,vcenter=vcenter.coolcorp.

I tested fadjar340’s configuration…: the result is the same
I only have a small portion of the data that displays

pmhausen · June 13, 2020, 6:13pm

With the result of the show series at hand, check the query section of your panels and look for mismatches …
One panel at a time, I am not super experienced with Grafana, yet, but that’s what I would do. Ready-made dashboards have so many implicit assumptions about how your collectd/telegraf and your influx/prometheus/whatever are configured, I found none of them working for my setup (FreeNAS/FreeBSD with collectd as the source).

bvivi57130 · June 13, 2020, 6:51pm

I think I’ve found the solution to my problem

This is a version issue with InfluxDB.

With Ubuntu 20.04, the repo “InfluxData - Package Repository” does not contain a definition for InfluxDB. So I use the version provided in the official Ubuntu repo, which is the 1.6.4 version.

If I use the repo “InfluxData - Package Repository” which corresponds to the previous version of ubuntu, I can install the version 1.8.0 of influxdb.

Once this version deployed, I have all the values in my dashboard

zoheirop · February 22, 2021, 3:34pm

hi,

did you make a downgrad of influxdb ?

vqmanh · September 8, 2022, 4:06am

Hello, I used InfluxDB 1.8, but It’s not working

alexandrearmand · September 8, 2022, 7:51am

Please upgrade to influxdb 2.4 +
this is much better !