It is there:
Grafana may offer you Dimension names or their values in the dropdown - but you can still write own if they are not correct: AWS Grafana GameLift Hard Coded Metrics - #2 by jangaraj
If you make a close look into your “working” metrics in the CloudWatch, then you will find that each of them has some dimensions - that’s what you need. Please note that some metrics may need multiple dimensions, not just one.
I would recommend to start with some ready RDS dashboard, e.g.
and then customize it for your need.