Contact point won't render template

Grafana Alerting
,
1

I’ve manually created a contact point for Slack with template that works fine. Unfortunately it doesn’t render title or text body when I created the same in Terraform.

In my terraform I have:

resource "grafana_contact_point" "contact_point" {
  name = "Terraform test"

  slack {
    title    = "{{ template \"slack.title\" . }}"
    text     = grafana_message_template.slack_template.template
    url      = var.slack_webhook
    username = "Grafana"
  }
}

resource "grafana_message_template" "slack_template" {
  name     = "Terraform test"
  template = <<EOT
{{ define "alert_severity_prefix_emoji" -}}
	{{- if ne .Status "firing" -}}
		:white_check_mark:
	{{- else if eq .CommonLabels.severity "critical" -}}
		:red_circle:
	{{- else if eq .CommonLabels.severity "warning" -}}
		:warning:
	{{- end -}}
{{- end -}}

{{ define "slack.title" -}}
  {{ template "alert_severity_prefix_emoji" . }}
	[{{- .Status | toUpper -}}{{- if eq .Status "firing" }} x {{ .Alerts.Firing | len -}}{{- end }}]: {{ .CommonLabels.alertname -}}
{{- end -}}

{{- define "slack.text" -}}
{{- range .Alerts -}}
{{ if gt (len .Annotations ) 0 }}
*Description*: {{ .Annotations.description }}
*Instance*:
{{ range .Labels.SortedPairs }}{{ if or (eq .Name "env") (eq .Name "instance") }}• {{ .Name }}: `{{ .Value }}`
{{ end }}{{ end }}
*Silence alert*: {{ .SilenceURL }}
*Go to dashboard*: {{ .DashboardURL }}
*Go to panel directly*: {{ .PanelURL }}
{{ end }}
{{ end }}
{{ end }}
EOT
}

resource "grafana_notification_policy" "slack_notification_policy" {
  group_by      = ["..."]
  contact_point = grafana_contact_point.contact_point.name

  policy {
    matcher {
      label = "instance"
      match = "="
      value = "x"
    }
    contact_point = grafana_contact_point.contact_point.name
    group_by      = ["..."]
    continue      = false
  }

  policy {
    matcher {
      label = "instance"
      match = "="
      value = "y"
    }
    contact_point = "Slack" # <- Manually created
    group_by      = ["..."]
    continue      = false
  }
}

This is how it looks in the UI:

grafana_contact_point2
grafana_contact_point2551×785 73.2 KB

And this is what I am getting in Slack:
grafana_contact_point

2

Hi! I can see two problems with your Terraform:

  1. It looks like you are storing the template in the contact point, but you should be using the grafana_message_template resource to store templates and then execute these templates from the contact point, like you are doing with the title where you have "{{ template \"slack.title\" . }}"
  2. The text in your Slack contact point is not executing any templates, instead it just contains a number of template definitions
1 Like
3

That was it, thanks. Do you know why in my alert I am getting empty annotations?
Again something that works fine when configured manually doesn’t work when terraformed:

resource "grafana_rule_group" "my_alert_rule" {
  name             = "Terraform test"
  folder_uid       = grafana_folder.rule_folder.uid
  interval_seconds = 180
  org_id           = 1
  rule {
    name           = "Test rule"
    for            = "20m"
    condition      = "C"
    no_data_state  = "NoData"
    exec_err_state = "Alerting"
    annotations = {
      "Dashboard UID" = grafana_dashboard.cloudwatch_RDS.uid
      "Panel ID" = "4"
      "Description" = "{{ with $values }} x MariaDB instance has high CPU usage: {{ humanize $values.B.Value }}% {{ end }}"
    }
    labels = {
      "env" = "y"
      "instance" = "x"
    }
...

While labels are displayed, Dashboard UID, Panel ID as well as Description are empty:
Grafana_alert_empty

4

Found the issue. It should be:

      __dashboardUid__ = grafana_dashboard.cloudwatch_RDS.uid
      __panelId__      = "4"
      "description" = "{{ with $values }} instance has high CPU usage: {{ humanize $values.B.Value }}% {{ end }}"