Using latest Grafana for the first time. I have a large kafka setup to overhaul monitoring of. big sys with a huge number of topics, zookeeper, producer & consumer groups, jmx & burrow etc, with full details going to grafana via prometheus.
-
What are you trying to achieve?
Picking up grafana and its querying again, and i have successfully listed all topics, but i am first trying to map the kafka setup via all the details in Grafana. Specifically three things, the overall list of topics, which producers are mapped to which topic, and which consumers are matched to which topic, as oversight for kafka in general.
i dont necessarily always have access to the expanding kafka environment directly, which would be easy to query for the details i’m after. however, anything added to kafka is always being added with prometheus config for gathering all the info. ideally i’d like to have the kafka topic mappings always auto-reflected in grafana ie. new topic gets added with new producer & consumer groups, and the details are available in grafana
-
How are you trying to achieve it?
as im quite new to grafana querying, and still experimenting my basic query attempts via the query builder is finding all the topics okay, and i can see via some of topic based queries ive tried, the info coming in that its revealing a link to consumers at least. just wondering if the query language is enough map out the producers and consumers as above… either for a list output of the mappings, or to dashboard for the mappings of producers>topics & consumers>topics.
any help appreciated