Indeed Athena is a perfectly viable option, if not for this solution representing a temporary measure…
The idea would be at the earliest convinience to migrate this data source to a more structured and flexible design, in our case SnowFlake which already includes a wide variety of our organizational tenants allowing us to support a wider range of use cases.
It seemed counter productive to create our team’s exclusive and robust data lake solution using something like AWS Glue/Athena to then be scrapped in favor of another tool more inline with organizational pratices.
The idea to consume it through HTTP does intrigues me, seing it as a possible low hanging fruit although again from what I’ve gathered Grafana also does not support this natively and requires again the yesoreyeram-infinity-datasource plugin. Would need to verify if in Grafana Cloud being a SaaS platform allows us to provide and set its own IAM Role, I’m aware if utilizing a native datasource (e.g. CloudWatch or Athena) there would be seemless akthough with this open source plugin would not sure TBH our bullet-proof alternative would be to expose a MS to retrieve this data and provide it through HTTP to grafana via this plugin has its datasource, although I can already foresee some waves or a bombardment of questions around data exposure even if we were to assure encryption on travel and propper authorization in place.
