Need some input on a complicated collection

Hello everyone,

I am currently debating between different tools to migrate our In-house monolith software that is responsible for collecting metric data (and other things) from over 10000 network devices.

One of the key points we need to be able to do is chained lookups. I found a nice post explaining this in the following link that will explain what I’m talking about: Support chained lookups · Issue #405 · prometheus/snmp_exporter · GitHub

I am not looking for an answer on how to do this collection, I just need a direction to documentation to make sure that this is something that can be done in a modern and robust way, for example python scripts that can be part of CI/CD pipeline etc etc.

From what I understand, I need an SNMP exporter configuration file to collect specific OIDs. Then add a job to scrape the results of this configuration file and produce graphs.

But how do you abstract all this to apply it to multiple devices/indexes at once?
A qos snmp question will generally contain the interface index, the queue number, the direction and the metric name. So X interfaces * 2 directions * 1-8 queue numbers * 16ish metrics is over 1000 metric points.

Can you do this kind of chained lookup using the SNMP exporter configuration file? Is there another way?

Any ideas or information about similar implementations will be highly appreciated.