500 Internal Server Error when trying to delete File Provisioned Alerts

  • What Grafana version and what operating system are you using?
    Windows Server 2019 v. 1809 build 17763.6054
    Grafana v11.1.3 (beac3bdbcb)

  • What are you trying to achieve?
    I’m trying to delete File Provisioned Alerts, basically I created a rule group manually, exported it, programatically changed the query information (queries to InfluxDB, same queries, different bucket) and uids, put that json on the conf/provisioning/alerting and now, the original rule group created manually was expanded, and I want to delete the provisioned one or disable provenance to either modify it manually or delete it altogether and establish a more “automated” provisioning flow

  • How are you trying to achieve it?
    I created a node.js script that fetches all the Alert Rules under a FOLDER and RULE-GROUP using:

I fetched and identified the uids of folders that I need and the rule group is “10s”

  for (const folderUid of folderUids) {
    const request = await grafanaApi.get(`/folders/${folderUid}`);
    console.log(request.data);
  }

then I tried to send an empty PUT request like the documentation says here:

    const response = await grafanaApiProvisioning.put(
        `/folder/${folderUid}/rule-groups/${group}`,
        {} // Sending empty data as specified
      );

I also tried deleting the group:

  for (const folderUid of folderUids) {
    try {
      const response = await grafanaApiProvisioning.delete(
        `/folder/${folderUid}/rule-groups/${group}`,
      );

      console.log(`Successfully delete folder and rule group!`);
    } catch (error) {
      console.error(JSON.stringify(error, null, 2));
    }
  }
  • What happened?
    For the updating FOLDER and RULE GROUP case:
PS C:\projects\alerts.replaced.for.privacy.com> node .\alertUpdateWithEmpty.js
{
  "message": "Request failed with status code 400",
  "name": "AxiosError",
  "stack": "AxiosError: Request failed with status code 400\n    at settle (file:///C:/projects/alerts.replaced.for.privacy.com/node_modules/axios/lib/core/settle.js:19:12)\n
   at IncomingMessage.handleStreamEnd (file:///C:/projects/alerts.replaced.for.privacy.com/node_modules/axios/lib/adapters/http.js:599:11)\n    at IncomingMessage.emit (node:e
vents:531:35)\n    at endReadableNT (node:internal/streams/readable:1696:12)\n    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)\n
at Axios.request (file:///C:/projects/alerts.replaced.for.privacy.com/node_modules/axios/lib/core/Axios.js:45:41)\n    at process.processTicksAndRejections (node:internal/proc
ess/task_queues:95:5)\n    at async sendEmptyDataPutRequest (file:///C:/projects/alerts.replaced.for.privacy.com/alertUpdateWithEmpty.js:32:26)",
  "config": {
    "transitional": {
      "silentJSONParsing": true,
      "forcedJSONParsing": true,
      "clarifyTimeoutError": false
    },
    "adapter": [
      "xhr",
      "http",
      "fetch"
    ],
    "transformRequest": [
      null
    ],
    "transformResponse": [
      null
    ],
    "timeout": 0,
    "xsrfCookieName": "XSRF-TOKEN",
    "xsrfHeaderName": "X-XSRF-TOKEN",
    "maxContentLength": -1,
    "maxBodyLength": -1,
    "env": {},
    "headers": {
      "Accept": "application/json, text/plain, */*",
      "Content-Type": "application/json",
      "Authorization": "Bearer SHOULDIREMOVETHISFORPRIVACY?WELLJUSTINCASE123456789",
      "X-Disable-Provenance": "true",
      "User-Agent": "axios/1.7.5",
      "Content-Length": "2",
      "Accept-Encoding": "gzip, compress, deflate, br"
    },
    "baseURL": "http://localhost:3000/api/v1/provisioning",
    "method": "put",
    "url": "/folder/advmtfqabm3uof/rule-groups/10s",
    "data": "{}"
  },
  "code": "ERR_BAD_REQUEST",
  "status": 400
}
{
  "message": "Request failed with status code 400",
  "name": "AxiosError",
  "stack": "AxiosError: Request failed with status code 400\n    at settle (file:///C:/projects/alerts.replaced.for.privacy.com/node_modules/axios/lib/core/settle.js:19:12)\n
   at IncomingMessage.handleStreamEnd (file:///C:/projects/alerts.replaced.for.privacy.com/node_modules/axios/lib/adapters/http.js:599:11)\n    at IncomingMessage.emit (node:e
vents:531:35)\n    at endReadableNT (node:internal/streams/readable:1696:12)\n    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)\n
at Axios.request (file:///C:/projects/alerts.replaced.for.privacy.com/node_modules/axios/lib/core/Axios.js:45:41)\n    at process.processTicksAndRejections (node:internal/proc
ess/task_queues:95:5)\n    at async sendEmptyDataPutRequest (file:///C:/projects/alerts.replaced.for.privacy.com/alertUpdateWithEmpty.js:32:26)",
  "config": {
    "transitional": {
      "silentJSONParsing": true,
      "forcedJSONParsing": true,
      "clarifyTimeoutError": false
    },
    "adapter": [
      "xhr",
      "http",
      "fetch"
    ],
    "transformRequest": [
      null
    ],
    "transformResponse": [
      null
    ],
    "timeout": 0,
    "xsrfCookieName": "XSRF-TOKEN",
    "xsrfHeaderName": "X-XSRF-TOKEN",
    "maxContentLength": -1,
    "maxBodyLength": -1,
    "env": {},
    "headers": {
      "Accept": "application/json, text/plain, */*",
      "Content-Type": "application/json",
      "Authorization": "Bearer SHOULDIREMOVETHISFORPRIVACY?WELLJUSTINCASE123456789",
      "X-Disable-Provenance": "true",
      "User-Agent": "axios/1.7.5",
      "Content-Length": "2",
      "Accept-Encoding": "gzip, compress, deflate, br"
    },
    "baseURL": "http://localhost:3000/api/v1/provisioning",
    "method": "put",
    "url": "/folder/advmtfqu0ta0wa/rule-groups/10s",
    "data": "{}"
  },
  "code": "ERR_BAD_REQUEST",
  "status": 400
}

weird because performing a GET from postman to api/v1/provisioning/alert-rules/:uid returns:

{
    "id": 33,
    "uid": "c93da1ee525c46",
    "orgID": 1,
    "folderUID": "advmtfqabm3uof",
    "ruleGroup": "10s",
    "title": "EDITEDFORPRIVACY",
    "condition": "C",
    "updated": "2024-08-28T15:39:47-06:00",
    "noDataState": "NoData",
    "execErrState": "Error",
    "for": "5m",
    "annotations": {
        "__dashboardUid__": "educfr0nomjggb",
        "__panelId__": "2"
    },
    "provenance": "file",
    "isPaused": false,
    "notification_settings": {
        "receiver": "EDITEDFORPRIVACY"
    },
    "record": null,
    "data": //removed to reduce space in post

}

For the deleting FOLDER and RULE GROUP case:

PS C:\projects\alerts.replaced.for.privacy.com> node .\removeProvisionedFromFolderGroup.js
{
  "message": "Request failed with status code 500",
  "name": "AxiosError",
  "stack": "AxiosError: Request failed with status code 500\n    at settle (file:///C:/projects/alerts.replaced.for.privacy.com/node_modules/axios/lib/core/settle.js:19:12)\n
   at IncomingMessage.handleStreamEnd (file:///C:/projects/alerts.replaced.for.privacy.com/node_modules/axios/lib/adapters/http.js:599:11)\n    at IncomingMessage.emit (node:e
vents:531:35)\n    at endReadableNT (node:internal/streams/readable:1696:12)\n    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)\n
at Axios.request (file:///C:/projects/alerts.replaced.for.privacy.com/node_modules/axios/lib/core/Axios.js:45:41)\n    at process.processTicksAndRejections (node:internal/proc
ess/task_queues:95:5)\n    at async deleteRuleGroup (file:///C:/projects/alerts.replaced.for.privacy.com/removeProvisionedFromFolderGroup.js:49:24)",
  "config": {
    "transitional": {
      "silentJSONParsing": true,
      "forcedJSONParsing": true,
      "clarifyTimeoutError": false
    },
    "adapter": [
      "xhr",
      "http",
      "fetch"
    ],
    "transformRequest": [
      null
    ],
    "transformResponse": [
      null
    ],
    "timeout": 0,
    "xsrfCookieName": "XSRF-TOKEN",
    "xsrfHeaderName": "X-XSRF-TOKEN",
    "maxContentLength": -1,
    "maxBodyLength": -1,
    "env": {},
    "headers": {
      "Accept": "application/json, text/plain, */*",
      "Content-Type": "application/json",
      "Authorization": "Bearer SHOULDIREMOVETHISFORPRIVACY?WELLJUSTINCASE123456789",
      "X-Disable-Provenance": "true",
      "User-Agent": "axios/1.7.5",
      "Accept-Encoding": "gzip, compress, deflate, br"
    },
    "baseURL": "http://localhost:3000/api/v1/provisioning",
    "method": "delete",
    "url": "/folder/advmtfqabm3uof/rule-groups/10s"
  },
  "code": "ERR_BAD_RESPONSE",
  "status": 500
}
{
  "message": "Request failed with status code 500",
  "name": "AxiosError",
  "stack": "AxiosError: Request failed with status code 500\n    at settle (file:///C:/projects/alerts.replaced.for.privacy.com/node_modules/axios/lib/core/settle.js:19:12)\n
   at IncomingMessage.handleStreamEnd (file:///C:/projects/alerts.replaced.for.privacy.com/node_modules/axios/lib/adapters/http.js:599:11)\n    at IncomingMessage.emit (node:e
vents:531:35)\n    at endReadableNT (node:internal/streams/readable:1696:12)\n    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)\n
at Axios.request (file:///C:/projects/alerts.replaced.for.privacy.com/node_modules/axios/lib/core/Axios.js:45:41)\n    at process.processTicksAndRejections (node:internal/proc
ess/task_queues:95:5)\n    at async deleteRuleGroup (file:///C:/projects/alerts.replaced.for.privacy.com/removeProvisionedFromFolderGroup.js:49:24)",
  "config": {
    "transitional": {
      "silentJSONParsing": true,
      "forcedJSONParsing": true,
      "clarifyTimeoutError": false
    },
    "adapter": [
      "xhr",
      "http",
      "fetch"
    ],
    "transformRequest": [
      null
    ],
    "transformResponse": [
      null
    ],
    "timeout": 0,
    "xsrfCookieName": "XSRF-TOKEN",
    "xsrfHeaderName": "X-XSRF-TOKEN",
    "maxContentLength": -1,
    "maxBodyLength": -1,
    "env": {},
    "headers": {
      "Accept": "application/json, text/plain, */*",
      "Content-Type": "application/json",
      "Authorization": "Bearer SHOULDIREMOVETHISFORPRIVACY?WELLJUSTINCASE123456789",
      "X-Disable-Provenance": "true",
      "User-Agent": "axios/1.7.5",
      "Accept-Encoding": "gzip, compress, deflate, br"
    },
    "baseURL": "http://localhost:3000/api/v1/provisioning",
    "method": "delete",
    "url": "/folder/advmtfqu0ta0wa/rule-groups/10s"
  },
  "code": "ERR_BAD_RESPONSE",
  "status": 500
}

For the one by one case:

{
  "message": "Request failed with status code 500",
  "name": "AxiosError",
  "stack": "AxiosError: Request failed with status code 500\n    at settle (file:///C:/projects/alerts.grupocayala.com/node_modules/axios/lib/core/settle.js:19:12)\n
   at IncomingMessage.handleStreamEnd (file:///C:/projects/alerts.grupocayala.com/node_modules/axios/lib/adapters/http.js:599:11)\n    at IncomingMessage.emit (node:e
vents:531:35)\n    at endReadableNT (node:internal/streams/readable:1696:12)\n    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)\n
at Axios.request (file:///C:/projects/alerts.grupocayala.com/node_modules/axios/lib/core/Axios.js:45:41)\n    at process.processTicksAndRejections (node:internal/proc
ess/task_queues:95:5)\n    at async deleteAllRulesInFolderAndRuleGroup (file:///C:/projects/alerts.grupocayala.com/removeProvisionedFromFolderGroup.js:96:23)",
  "config": {
    "transitional": {
      "silentJSONParsing": true,
      "forcedJSONParsing": true,
      "clarifyTimeoutError": false
    },
    "adapter": [
      "xhr",
      "http",
      "fetch"
    ],
    "transformRequest": [
      null
    ],
    "transformResponse": [
      null
    ],
    "timeout": 0,
    "xsrfCookieName": "XSRF-TOKEN",
    "xsrfHeaderName": "X-XSRF-TOKEN",
    "maxContentLength": -1,
    "maxBodyLength": -1,
    "env": {},
    "headers": {
      "Accept": "application/json, text/plain, */*",
      "Content-Type": "application/json",
      "Authorization": "Bearer SHOULDIREMOVETHISFORPRIVACY?WELLJUSTINCASE123456789",
      "X-Disable-Provenance": "true",
      "User-Agent": "axios/1.7.5",
      "Accept-Encoding": "gzip, compress, deflate, br"
    },
    "baseURL": "http://localhost:3000/api/v1/provisioning",
    "method": "delete",
    "url": "/alert-rules/a3a69e41bdbe44"
  },
  "code": "ERR_BAD_RESPONSE",
  "status": 500
}
{
  "message": "Request failed with status code 500",
  "name": "AxiosError",
  "stack": "AxiosError: Request failed with status code 500\n    at settle (file:///C:/projects/alerts.grupocayala.com/node_modules/axios/lib/core/settle.js:19:12)\n
   at IncomingMessage.handleStreamEnd (file:///C:/projects/alerts.grupocayala.com/node_modules/axios/lib/adapters/http.js:599:11)\n    at IncomingMessage.emit (node:e
vents:531:35)\n    at endReadableNT (node:internal/streams/readable:1696:12)\n    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)\n
at Axios.request (file:///C:/projects/alerts.grupocayala.com/node_modules/axios/lib/core/Axios.js:45:41)\n    at process.processTicksAndRejections (node:internal/proc
ess/task_queues:95:5)\n    at async deleteAllRulesInFolderAndRuleGroup (file:///C:/projects/alerts.grupocayala.com/removeProvisionedFromFolderGroup.js:96:23)",
  "config": {
    "transitional": {
      "silentJSONParsing": true,
      "forcedJSONParsing": true,
      "clarifyTimeoutError": false
    },
    "adapter": [
      "xhr",
      "http",
      "fetch"
    ],
    "transformRequest": [
      null
    ],
    "transformResponse": [
      null
    ],
    "timeout": 0,
    "xsrfCookieName": "XSRF-TOKEN",
    "xsrfHeaderName": "X-XSRF-TOKEN",
    "maxContentLength": -1,
    "maxBodyLength": -1,
    "env": {},
    "headers": {
      "Accept": "application/json, text/plain, */*",
      "Content-Type": "application/json",
      "Authorization": "Bearer SHOULDIREMOVETHISFORPRIVACY?WELLJUSTINCASE123456789",
      "X-Disable-Provenance": "true",
      "User-Agent": "axios/1.7.5",
      "Accept-Encoding": "gzip, compress, deflate, br"
    },
    "baseURL": "http://localhost:3000/api/v1/provisioning",
    "method": "delete",
    "url": "/alert-rules/de8184bfd25246"
  },
  "code": "ERR_BAD_RESPONSE",
  "status": 500
}
  • What did you expect to happen?
    File Provisioned Alerts to be deleted or at least removed that “Provisioned” tag to be able to modify it manually