Provisioning alerts, recording rules for Grafana 9 by helm chart

Grafana Alerting
, , ,
1

I’m using Grafana 9 managed by helm charts. Setting alerting manually trough UI is fine, but it would be much nicer if I could provision them by helm chart as well. Atm I’m using helm chart ver. 6.29.11 and moving to 6.31.2. Any advice? Or using API is the only way?

What would be the best practice in this case?

Thank you…

4 Likes
2

I am very curious about this too and I am searching information with days. :smiley: It would be great if someone knows.

1 Like
3

Did you solve it? I don’t understand why it is so difficult to find it:(((

4

Hello,

I did not. The only progress I have is that I got the alert’s JSON code via the API, but even if I manually set the file into /etc/grafana/provisioning/alerts (tried with both - json and yaml) , it does not appear on the interface.

5

Hello,

Any update?

BR!

6

Hi @dselena,

I posted this by mistake on a Grafana 8 thread. I can’t seem to get json provisioning working via the api on Grafana 9.3.1 on kubernetes (as a first step to provisioning via config files/sidecar).
Can you elaborate on how you managed to do it?

I created an alert in the UI and then exported it:

curl -XGET  http://some:password@somegrafana/api/v1/provisioning/alert-rules/1jIgdlKVk | jq . > alert-rule.json

I would expect to be able to import the exported rule, but when I delete the rule or delete the grafana pod and try and post back to the api I get this error:

curl -XPOST  http://some:password@somegrafana//api/v1/provisioning/alert-rules -d @alert-rule.json      
{"message":"bad request data","traceID":""}

there is nothing in the grafana log to expand on this error and cross checking the contents of the json with the api guide: Alerting Provisioning HTTP API | Grafana documentation
it seems to be ok…

unless the keys really need to be upper case, i.e. my json (returned by the api) starts:

{
  "id": 1,
  "uid": "TSKdFlF4k",
  "orgID": 1,

but in the api docs, its “ID”, “UID” etc.
Thanks,
Tom

7

Hi @thopewell

if I use the template for the body from here I also get the same error. (That body might be outdated. I will reach out internally about that.)

So, I basically did the same thing. I manually created an alert rule via the UI, like you did (I used TestData as its datasource)
Then I GOT the alert JSON via the /api/v1/provisioning/alert-rules/{UID} endpoint

Finally I used that JSON to POST a new rule (but incrementing the id, leaving the uid empty, and giving it a new title)

{
    "id": 48,
    "uid": "",
    "title": "mytest222",
}

Hope it works for you

8

but when I delete the rule or delete the grafana pod and try and post back to the api I get this error

Make sure to set the Content-Type: application/json header. You can get bad request data if you send JSON without telling Grafana that the request contains JSON.