I have spent a considerable amount of time attempting to create a dashboard which shows multiple metrics from multiple nodes all at once. This does not seem possible with Prometheus and the panel plugins that are available now but if I am incorrect please let me know.
I have configured and I am using templating and variables which is getting me closer however there is no dependency between the variables. If I have two variables $host and $ip referenced in multiple places in a panel, say SingleStat. If I choose a particular ip, the values of the panel are not changed to reference the corresponding value of the $host. Hope that makes sense.
SingleStat is almost there, but the obvious limitation of just displaying a single stat is not going to work in this case.
The StatusPanel is also very close but like everything else it accomplishes 90% of what I need but fails in one area. It cannot handle Prometheus labels in the metrics from best I can tell. Officially it is not even supported for Prometheus but still works for most options. The issue in my case is that I have a snmp result with a string that I need to display, but need to use a variable obviously so the panels can be repeated using templating. In the metric itself this is not supported unlike the SingleStat where this can be accomplished using the “Named” option.
I can’t believe I am the only one out here using Prometheus and attempting to design a dashboard with multiple nodes/metrics on a single screen. I have scoured the published Dashboards on the official site and I could not find any that accomplish this.
If anyone has any input or advice on this issue I would appreciate it.