Hi!
-
What Grafana version and what operating system are you using?
I am using Grafana 8.0.4 on ubuntu 20.04.2 LTS, running on a odroid C2 (single board computer, similar to raspberry pi) -
What are you trying to achieve?
I am trying to make grafana send alarms to my signal messenger account. -
How are you trying to achieve it?
- enabled ngAlerts (seems to be working)
- configured a contact point: Webhook.
- set up a default notification policy
There, I set up custom timings:
Group wait = 1s
Group interval =1s
Repeat interval = 1d
The webhook that forwards the message to signal-cli is based on:
I added a schema for the new json fromat of the ngAlerts.
- What happened?
I sucessfully received altert messes on my signal account. However, a message for one alert is send by Grafana every ~30s, unless i mute the alarm.
- What did you expect to happen?
As the repeat interval is set to 1d, I was expecting to receive a message only after another day, or in case that the alarm condition was not fulfilled for a rime and then fulfilled again (which was not the case).
- Can you copy/paste the configuration(s) that you are having problems with?
I guess I don’t really understand how the webhook works / what the format of the json object is. Also I was not able to find any proper documentation for it.
Maybe the webhook contact point awaits a confirmation on the sent message?
I tried to build a schema from an alert json message sent by grafana. The schema looks like this:
class NGALabels(BaseModel):
alertname: str
rule_uid: str
class NGAAnnotations(BaseModel):
message: str
class NGAlert(BaseModel):
status: str
labels: NGALabels
annotations: NGAAnnotations
startsAt: str
endsAt: str
generatorURL: str
fingerprint: str
silenceURL: str
dashboardURL: str
panelURL: str
valueString: str
class NGAGrafanaOutgoing(BaseModel):
receiver: str
status: str
alerts: List[NGAlert]
groupLabels: Dict[str, Any]
commonLabels: Dict[str, Any]
commonAnnotations: Dict[str, Any]
externalURL: str
version: str
groupKey: str
truncatedAlerts: int
title: str
state: str
message: str
class MessageSentNGAGrafana(NGAGrafanaOutgoing):
timestamp: datetime
the message is generated here:
@router.post("{number}/ngagrafana/", response_model=MessageSentNGAGrafana, status_code=201)
async def send_message(
message: NGAGrafanaOutgoing, number: str, background_tasks: BackgroundTasks,
receiver: str, group: bool = True,
) -> Any:
"""
send message
"""
message_string = "numebr of alerts: "+ str(len(message.alerts)) + "\n"
i=0
#print ( "numebr of alerts: ", len(message.alerts))
for alert in message.alerts:
i=i+1
message_string += "#" + str(i) + ":\n"
message_string += alert.status + "\n"
message_string += alert.labels.alertname + "\n"
message_string += alert.annotations.message + "\n"
message_string += alert.silenceURL + "\n"
message_string += alert.valueString + "\n\n"
"""
if message.evalMatches is not None:
for match in message.evalMatches:
metrics += "\n" + match.metric + ": " + str(match.value)
message_string = message.title + "\n" + "State: " + message.ruleName + "\n" + "Message: " + message.message + "\n" + "URL: " + message.ruleUrl + "\n\n" + "Metrics: " + metrics
"""
cmd = ["-u", quote(number), "send", "-m", quote(message_string)]
receivers = []
if group:
cmd.append("-g")
cmd.append(quote(receiver))
else:
receivers.append(receiver)
cmd += list(map(quote, receivers))
response = await run_signal_cli_command(cmd)
print ("response: ", response)
return MessageSentNGAGrafana(**message.dict(), timestamp=response.split("\n")[0])
at the very end, I think the confirmation is sent to grafana. if i understand it correctly, that is just the same message, extended by the timestamp of the execution.
- Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.
none that I am aware of - Did you follow any online instructions? If so, what is the URL?
The webhook I modified is from:
GitHub - kahrpatrick/signal-cli-rest-api: signal-cli-rest-api is a wrapper around signal-cli and allows you to interact with it through http requests
the modified files are:
signal_cli_rest_api/schemas.py
signal_cli_rest_api/api/messages.py
Can I just send one message once the alert condition is fulfilled?
Or did I understand the concept of the new altertin incorrectly?
Should I use a different apporoach?
Best regards
Christian