What Grafana version and what operating system are you using?
Grafana v11.5.1 inside of Docker
What are you trying to achieve?
Using Prometheus as a datasource, combine data from multiple rows into a single row.
For some background, I’m using 2 different Prometheus exporters.
One tracks Slurm GPU indexes and reports whether or not it is allocated to a job.
slurm_node_gpu_alloc{index="0", instance="x:8080", job="slurm", node="gpu06", type="a100"} 1
slurm_node_gpu_alloc{index="1", instance="x:8080", job="slurm", node="gpu06", type="a100"} 1
slurm_node_gpu_alloc{index="2", instance="x:8080", job="slurm", node="gpu06", type="a100"} 0
slurm_node_gpu_alloc{index="3", instance="x:8080", job="slurm", node="gpu06", type="a100"} 0
The second tracks actual GPU usage, though instead of by ID it reports based off of UUID.
nvidia_smi_utilization_gpu_ratio{instance="gpu06:9835", job="gpu", uuid="72c65b2b-c87f-9607-25ad-66d250b24ea1"} 0.04
nvidia_smi_utilization_gpu_ratio{instance="gpu06:9835", job="gpu", uuid="55872738-ecd1-0a14-2e1d-745ef4b99fcb"} 0.95
nvidia_smi_utilization_gpu_ratio{instance="gpu06:9835", job="gpu", uuid="56d82bbe-49c8-0ca6-2be7-4721921038a4"} 0
nvidia_smi_utilization_gpu_ratio{instance="gpu06:9835", job="gpu", uuid="157efaca-8b9e-ebe7-d2af-0e8b3b55651a"} 0.23
There is another metric exported by the second exporter which also reports the GPU index in the system
nvidia_smi_index{instance="gpu06:9835", job="gpu", uuid="72c65b2b-c87f-9607-25ad-66d250b24ea1"} 0
nvidia_smi_index{instance="gpu06:9835", job="gpu", uuid="55872738-ecd1-0a14-2e1d-745ef4b99fcb"} 1
nvidia_smi_index{instance="gpu06:9835", job="gpu", uuid="56d82bbe-49c8-0ca6-2be7-4721921038a4"} 2
nvidia_smi_index{instance="gpu06:9835", job="gpu", uuid="157efaca-8b9e-ebe7-d2af-0e8b3b55651a"} 3
Using the above 3 metrics, I want to be able to create a table that will list the machine name, index, UUID, if it’s allocated, and utilization. For example, with the above data create the following table:
| Machine Name | Index | UUID | Allocated? | Utilization |
|---|---|---|---|---|
| gpu06 | 0 | 72c65b2b-c87f-9607-25ad-66d250b24ea1 | 1 | 0.410 |
| gpu06 | 1 | 55872738-ecd1-0a14-2e1d-745ef4b99fcb | 1 | 0 |
| gpu06 | 2 | 56d82bbe-49c8-0ca6-2be7-4721921038a4 | 0 | 0 |
| gpu06 | 3 | 157efaca-8b9e-ebe7-d2af-0e8b3b55651a | 0 | 0 |
What have I tried
My current queries consist of the following:
Along with the following Transformations:
And I end up with the following table:
It’s close, but I believe that it needs to join the “index” and “Value #B” columns, and “Value #B” and “uuid” columns. I just cannot for the life of me figure it out.
Any help would be appreciated, if it’s even possible.