Dashboard not loading Dimension Values from CloudWatch Data Source

I’m using Grafana as our main observability tool for monitoring resources in AWS via AWS Managed Grafana.

We are using AWS Managed Grafana in 2 AWS Accounts: 1 for production and 1 for dev.

We have also set-up CloudWatch cross-account observability, having Cloudwatch from dev reporting to prod, as to make devs logs available to prod.

First we set-up all of our dashboards in dev and everything is fine, but when trying to set up the same dashboards in prod, it is just not loading the dimension values for some services, making them useless to us as it does not show data, or at least the way we wanted and previously worked on the dev environment.

The cloudwatch metrics we want to show in prod are from the prod account + the ones from dev. The issue we are having are from the resources from prod itself, not being shown in the grafana instance for prod.

To get the issue: We are trying to set a dashboard to monitor the ELB in the prod account, following what we did in the dev account using grafana’s and AWS documentation. Below is the expected result of the dev account.


Now, when we do this on the prod account these variables results are not loading. See below image for the prod account:


And when we navigate to cloudwatch in the AWS account for prod and look for the ApplicationELB namespace (below image), we clearly see the that the data exists and its there, but it’s not reachable? from Grafana?

We tried all sort of stuff but me and my team are new to grafana, so some ideas into why this is happening would be greatly appreciated.

Any extra info or logs we would need to provide pls let us know. Thanks!!

I would start with ready dashboards, which are working - https://github.com/monitoringartist/grafana-aws-cloudwatch-dashboards/tree/master/aws-elb You can customize them for your needs.

Yes, I was actually using a template already. This template I got it from the Grafana labs page (id is 650 for the ELB one) fully worked on my dev environment, showing the dimension values on the variable

The issue is, when using the same template, and configuration on both Grafana and cloudwatch and other AWS resources (correct IAM roles and permission for example) on my prod environment, the dimension values are not showing on the dashboard

You are not using the same “template” = dashboard, because $region != $Region. You used also different metric. So it looks like you modified dashboard and that can be source of your problem.

That dashboard was never tested with cross observability. It is AWS managed, so just ask your AWS support for that.

Oh, I’m sorry about that!

I was toying around with the options to see if I could get anything out of it. I’m out of the office so tomorrow I can follow up with the proper screenshots, showing identical configurations.