Best way to automatically provision notification channel?

I’m trying to figure out the best way to provision notification channels automatically so that I can keep track of them in source control.

I saw there is yaml support for doing the provisioning with datasources but couldn’t find anything for notification channels but did see that it is possible through the API.

What is the best way to add notification channels? Create a json file and just tell Grafana to look for that config?

1 Like

Even I am looking for a way to embed default Notification Channels into the Docker Image. Yet to find a way. Will be asking it in the forum to get some advice/assistance.

– Sarat

1 Like

No reply since?

FYI, just to be clear the API would be this I believe: http://docs.grafana.org/http_api/alerting/#create-alert-notification

So @jmreicha, I guess you could commit the JSON(s) used by the API to source control and simply invoke API when provisioning/deploying server from scratch?

And @saratrajan, did you get an answer to your question? If not, could you not use the API, calling it during docker build as steps in the Dockerfile to create/embed the notification channels setup, and for the JSON input, have that in a file in the docker image’s source repo path so that it can be used by the docker build process in the Dockerfile. This of course would be after the server is installed and brought up during image build process to then import dashboards and set up the notification channels. So future invocations of image should start up with grafana preconfigured.

@daluu I don’t suppose you have an example of how to do this that you could share?

I could share a generalized version of what we have whenever it’s implemented, I just started looking into this. But I would think implementation is somewhat specific to the provisioning platform and the deployment/runtime platform (e.g. docker, vagrant, terraform, just a plain server box, AWS EC2 instance, kubernetes, chef, ansible, puppet, shell scripting, etc.) in regards to where/when/how you execute the API call provisioning step, and how you make the API call (what tool/library to do the call).

I took a shot at it looking further with what we were doing at our organization, and here’s a proof of concept demo. It’s not complete for the actual alerting to work (email, slack), nor does it have real dashboard with alert to demo. But it at least shows that the alert notification channel was created in the build image that you can then just deploy and have it ready to go.

particularly the sample config changes starting here: https://github.com/daluu/grafana-config-customizations/blob/master/install/install.sh#L20, with the API request body JSON files in the same folder.

Hope that helps. This of course is for docker based deployments. But the sample is generic enough you can adapt for other platforms.

This is great, thanks for sharing!

I looked into things further at my workplace, we use terraform, in some cases combined with docker like the grafana server part. If using terraform, you also have this option to configure the alert notification channels: https://www.terraform.io/docs/providers/grafana/r/alert_notification.html

However, the provider for terraform does not appear to allow you to configure the prerequisites of SMTP mail server details and external image storage and grafana URL/domain.

https://grafana.com/docs/grafana/latest/administration/provisioning/

Seems to explain pretty well how to provision a notification channel, I’ve just tested and this works.

1 Like