How configure kafka for accept all notification alert in using a notification channels( kafka REST proxy)

Grafana Alerting
, ,
1

hello,
I want push my alert into kafka . I can’t find how to do it…

We observe exceptions while testing the notification

grafana | logger=alerting.notifier t=2022-09-20T08:48:34.84+0000 lvl=eror msg=“failed to send notification” uid= error=“Post "http://0.0.0.0:29092/topics/audio_alert\”: dial tcp 0.0.0.0:29092: connect: connection refused"
grafana | logger=alerting.notifier t=2022-09-20T08:48:34.84+0000 lvl=eror msg=“failed to send notification” uid= error=“Post "http://0.0.0.0:29092/topics/audio_alert\”: dial tcp 0.0.0.0:29092: connect: connection refused"
grafana | logger=context t=2022-09-20T08:48:34.84+0000 lvl=eror msg=“Failed to send alert notifications” error=“Post "http://0.0.0.0:29092/topics/audio_alert\”: dial tcp 0.0.0.0:29092: connect: connection refused" remote_addr=172.29.0.1
grafana | logger=context t=2022-09-20T08:48:34.84+0000 lvl=eror msg=“Request Completed” method=POST path=/api/alert-notifications/test status=500 remote_addr=172.29.0.1 time_ms=12 size=48 referer=http://0.0.0.0:3000/alerting/notification/2/edit
grafana | logger=alerting.evalContext t=2022-09-20T08:48:36.7+0000 lvl=info msg=“Alert Rule returned no data” ruleId=1 name=“GPU Temperature alert” changingstateto=no_data
grafana | logger=alerting.notifier.kafka t=2022-09-20T08:48:40.71+0000 lvl=info msg=“Notifying Kafka” alert_state=alerting
kafka_cp_1 | [2022-09-20 08:48:40,713] WARN [SocketServer brokerId=1] Unexpected error from /172.29.0.9; closing connection (org.apache.kafka.common.network.Selector)
kafka_cp_1 | org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 1347375956 larger than 1195725856)

|kafka_cp_1 | |at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:132)|
|—|—|
|kafka_cp_1 | |at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:93)|
|kafka_cp_1 | |at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:244)|
|kafka_cp_1 | |at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:205)|
|kafka_cp_1 | |at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:557)|
|kafka_cp_1 | |at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:495)|
|kafka_cp_1 | |at org.apache.kafka.common.network.Selector.poll(Selector.java:424)|
|kafka_cp_1 | |at kafka.network.Processor.poll(SocketServer.scala:665)|
|kafka_cp_1 | |at kafka.network.Processor.run(SocketServer.scala:582)|
|kafka_cp_1 | |at java.lang.Thread.run(Thread.java:748)|

2

Can you point us to a documentation you used or your alert config to kafka channel, how did you set it up. I notice xonnection refused in the errors, does it require authentication? Or does it need to be configured to accept notification from grafana ip?

3

[yosiasz,
" I notice connection refused in the errors " I was sending my alerts to the wrong url.

I continue my integration tests, providing a docker-compose of kafka’s configuration for this feature would be a good idea ?