We want to scrape topics from a kafka broker using Loki Promtail. Configured our kafka scrape like documented: Configure Promtail | Grafana Loki documentation
Initial startup does works, so kafka broker looks okay and my topics are available:
level=info ts=2024-09-30T07:46:06.858115234Z caller=target_syncer.go:246 msg=“new topics received” topics=[logs-topic-458]
level=info ts=2024-09-30T07:46:06.858131351Z caller=consumer.go:54 msg=“starting consumer” topics=[logs-topic-458]
level=info ts=2024-09-30T07:46:10.005022484Z caller=consumer.go:97 msg=“consuming topic” details=“member_id=sarama-881ec22a-54aa-4704-ad56-d2e345a80f6f generation_id=1 topic=logs-topic-458 partition=5 initial_offset=-2”
But after restarting Promtail the kafka consumer gets stuck with this error messages:
level=info ts=2024-09-30T07:51:43.848857442Z caller=target_syncer.go:246 msg=“new topics received” topics=[logs-topic-458]
level=info ts=2024-09-30T07:51:43.848872192Z caller=consumer.go:54 msg=“starting consumer” topics=[logs-topic-458]
level=error ts=2024-09-30T07:51:51.943328154Z caller=consumer.go:64 msg=“error from the consumer, retrying…” err=“kafka server: Request was for a consumer group that is not coordinated by this broker”
Promtail is using the Sarama library to connect to kafka. Do I have to set some properties for the Samara code to handle reconnects to kafka successful? Did anybody have similar issues?
Thanks, Andy