AWS Cloudwatch datasource with an assumed IAM role

Grafana Cloudwatch
,
1

Hi,

I have successfully configured Grafrana to access EC2 metrics using the following IAM role policy on the EC2 instance:

{
“Version”: “2012-10-17”,
“Statement”: [
{
“Sid”: “AllowReadingCloudwatchMetrics”,
“Effect”: “Allow”,
“Action”: [
“cloudwatch:PutMetricData”,
“cloudwatch:GetMetricStatistics”,
“cloudwatch:GetMetricData”,
“cloudwatch:ListMetrics”
],
“Resource”: “"
},
{
“Sid”: “AllowReadingTagsFromEC2”,
“Effect”: “Allow”,
“Action”: [
“ec2:DescribeTags”,
“ec2:DescribeInstances”
],
“Resource”: "”
}
]
}

This was working in our dev AWS account.

I tried then to allow Grafana to access the EC2 data in our prod AWS account by creating the same role policy as above in the prod account, and then allow the EC2 instance in the dev account to assume the role with this policy:

“Version”: “2012-10-17”,
“Statement”: [
{
“Sid”: “AssumeDashboardRole”,
“Effect”: “Allow”,
“Action”: “sts:AssumeRole”,
“Resource”: [
“arn:aws:iam:::role/dashboard”
]
}
]

This does not work. I noticed this https://github.com/grafana/grafana/issues/3522 which mentions manually calling STS.assumeRole and then adding the credentials to grafana configuration. Since the post is quite old and I didn’t need to add the credentials when using a single AWS account, I wondered if this was still necessary?

Thanks,
Andy.

2

Just noticed in the config there is a field for assumed role. Doh!

3

I have 6 AWS accounts and I’d like to be able to set up my Grafana instance to be able to access them (they are in different accounts from the account that Grafana is running in). This apparently done by setting up IAM roles and policies and setting up trust conditions.

I try assuming a role in another account and I still can’t get it to work. Unfortunately, Grafana isn’t returning very meaningful errors (I changed the log level to debug). It’s basically saying it can’t get the Metrics even those my IAM roles are using the Full-Access, Read-Only managed role for CloudWatch.

If you have an tips on how to set up multiple AWS accounts with the CloudWatch data plug-in I’d appreciate it!

1 Like
4

Grafana runs in AWS account A and you want to see CloudWatch data from AWS account B.

Run this CloudFormation template (replace <aws-account-A-id> with your AWS account A id) in AWS account B:

{
    "AWSTemplateFormatVersion": "2010-09-09",
    "Description": "AWS CloudFormation to create role for Grafana",
    "Resources": {
        "GrafanaRole": {
            "Type": "AWS::IAM::Role",
            "Properties": {
                "RoleName": "grafana-cloudwatch-role",
                "AssumeRolePolicyDocument": {
                    "Version": "2012-10-17",
                    "Statement": [{
                        "Effect": "Allow",
                        "Principal": {
                            "AWS": "arn:aws:sts::<aws-account-A-id>:root"
                        },
                        "Action": ["sts:AssumeRole"]
                    }]
                },
                "Policies": [{
                    "PolicyName": "GrafanaRole",
                    "PolicyDocument": {
                        "Version": "2012-10-17",
                        "Statement": [{
                                "Sid": "AllowReadingMetricsFromCloudWatch",
                                "Effect": "Allow",
                                "Action": [
                                    "cloudwatch:DescribeAlarmsForMetric",
                                    "cloudwatch:DescribeAlarmHistory",
                                    "cloudwatch:DescribeAlarms",
                                    "cloudwatch:ListMetrics",
                                    "cloudwatch:GetMetricStatistics",
                                    "cloudwatch:GetMetricData"
                                ],
                                "Resource": "*"
                            },
                            {
                                "Sid": "AllowReadingLogsFromCloudWatch",
                                "Effect": "Allow",
                                "Action": [
                                    "logs:DescribeLogGroups",
                                    "logs:GetLogGroupFields",
                                    "logs:StartQuery",
                                    "logs:StopQuery",
                                    "logs:GetQueryResults",
                                    "logs:GetLogEvents"
                                ],
                                "Resource": "*"
                            },
                           {
                                "Sid": "AllowReadingTagsInstancesRegionsFromEC2",
                                "Effect": "Allow",
                                "Action": [
                                    "ec2:DescribeTags",
                                    "ec2:DescribeInstances",
                                    "ec2:DescribeRegions"
                                ],
                                "Resource": "*"
                            },
                            {
                                "Sid": "AllowReadingResourcesForTags",
                                "Effect": "Allow",
                                "Action": "tag:GetResources",
                                "Resource": "*"
                            }
                        ]
                    }
                }]
            }
        }
    }
}

Required minimal policy is documented in AWS CloudWatch | Grafana Labs

This is just example CF template and you as admin of AWS account A should be able to know what you can accept, e.g. do you really want to allow arn:aws:sts::<aws-account-A-id>:root = all resources from AWS account A to assume that role? Maybe you will want to use also External ID to prevent the confused deputy problem.

This is advance AWS IAM topic, so it really not a good case to have it in the Grafana doc. AWS Amazon Managed Grafana is provided by AWS, so it is more rational, that you can find more details about advance IAM topics directly in the vendor (AWS) doc.

5

Thank you Jan!

This is what I’m looking for. This is similar to what I’ve tried so far, but I’ll compare your CFT to what I have.

Thanks again.

6

Generic CloudFormation template for cross account access to CloudWatch metric/logs - GitHub - monitoringartist/grafana-cross-account-cloudwatch-access: IAM role to allow Grafana read CloudWatch metrics/logs from another AWS account