Dear all,
I’m using cloudwatch plugin with cross-account observability. We are managing quite a few number of accounts (around 40) and hundreds of buckets (not all relevant to us). With default plugin configuration when I try to monitor bucket size the data I recieve varies every time I refresh the page. Like at some point AWS cloudwatch stops sending data for all the resources, so sometimes I see some buckets that go away on the next load.
Notice that in this two runs one 979TB bucket (among others) is gone from the picture. As said, I just refreshed the graphs.
Following advice from cloudwatch docs I ended up playing around adding qeuries for each an every account I am interested on and playing with the period value and, finally, found some conf that (I belive) works. The problem is that the max time range I can use is 7days. If I try to monitor buckets for more than that, I start to see the warning on “cloudwatch GetMetricData error: Too many datapoints requested - your search has been limited. Please try to reduce the time range”.
So, this means that I can only explore my bucket data for the last 7 days and I’m missing big part of “what happened to my buckets in the past” picture.
So I’m wondering how others are dealing with this problem when trying to monitor a bunch of S3 data. I did not find a way to limit the list of buckets I want to monitor (maybe all that start with certain prefix so I can drop anything cdk/terraform related) .
Thanks