Cloudwatch datasource default region not properly used in dashboard

Grafana Configuration
1

  • What Grafana version and what operating system are you using?
    9.4.3, grafana docker image on kubernetes

  • What are you trying to achieve?
    All our grafana configuration including datasources and dashboards is provisioned.
    The cloudwatch datasource gets a distinct region configured as default region.
    Dashboards get “default” as provisioned region, so we only have to configure the region once, centrally, and not set the distinct region in every dashboard configuration.

  • How are you trying to achieve it?

  • Can you copy/paste the configuration(s) that you are having problems with?

cloudwatch datasource provisioning snippet

 datasources:
      datasources.yaml:
        apiVersion: 1
        datasources:
        - name: CloudWatch
          type: cloudwatch
          uid: ds_cloudwatch
          editable: true
          jsonData:
            authType: default
            defaultRegion: $AWS_REGION

dashboard template snippet - variable “region” configuration with value “default”

  "templating": {
    "list": [
      {
        "current": {
          "selected": false,
          "text": "default",
          "value": "default"
        },
        "datasource": {
          "type": "cloudwatch",
          "uid": "$datasource"
        },
        "definition": "regions()",
        "hide": 0,
        "includeAll": false,
        "label": "Region",
        "multi": false,
        "name": "region",
        "options": [],
        "query": "regions()",
        "refresh": 1,
        "regex": "",
        "skipUrlSync": false,
        "sort": 0,
        "tagValuesQuery": "",
        "tagsQuery": "",
        "type": "query",
        "useTags": false
      },
  • What happened?
    Dashboard queries to the Cloudwatch datasource literally use region “default” to query Cloudwatch, which then returns empty results.
{
    "from": "1678283651212",
    "to": "1678294451212",
    "queries": [
        {
            "timezoneUTCOffset": "+0100",
            "intervalMs": 10800000,
            "maxDataPoints": 1,
            "alias": "",
            "datasource": {
                "uid": "ds_cloudwatch",
                "type": "cloudwatch"
            },
            "dimensions": {
                "Region": [
                    "default"
                ],
                "Rule": [
                    "*"
                ],
                "WebACL": [
                    "test"
                ]
            },
            "expression": "",
            "id": "",
            "matchExact": false,
            "metricEditorMode": 0,
            "metricName": "BlockedRequests",
            "metricQueryType": 0,
            "namespace": "AWS/WAFV2",
            "period": "",
            "queryMode": "Metrics",
            "refId": "B",
            "region": "eu-central-1",
            "sqlExpression": "",
            "statistic": "Sum",
            "label": "",
            "type": "timeSeriesQuery"
        }
    ]
}

  • What did you expect to happen?
    Dashboard queries to the Cloudwatch datasource resolve the “default” to the configured default region of the cloudwatch datasource, and query cloudwatch with that regions name.

  • Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.
    No error messages in the grafana logs or the browser developer tools, just empty data sets for literal region “default”.

So, my question is, how to properly configure a dashboard variable to use the datasources default region?