What is the right metric, useful to display when the last topic has arrived from Kafka?

I use Grafana version v10.2.2.

Is there a metric, using Thanos as data source, that allows to see if the last topic has arrived from Kafka or, conversely, how long has it been since we received a post per topic?

I used kafka_log_log_logendoffset, which is useful to obtain the maximum available offset in a Kafka partition. This represents the last available offset in the log of a partition. It’s possible to compare the maximum offset with the consumer offset to determine if new messages are present.

I’ll show you an example in my dashboard, divided by environment (plants, in this case) and topics from Kafka:

This is a time difference between the current time instant (time()) and the timestamp of the arrival of the last message for the first 10 topics, based on the sum of the increment rates of the maximum available offset in the 10-minute period. The resulting difference will be in duration format (e.g., seconds, minutes) and it will represent an approximation of the elapsed time since the last message arrived for the specified topics.

My questions are: why is there so much delay for every single topic, inside the environment? Is this metric working fine or should I employ some other metrics?

Thanks in advance :grinning: