Cross-tenant Prometheus query from Azure Managed Grafana without client secrets?

Grafana
, ,
1

Hi all,

I’m trying to set up an Azure Managed Grafana instance in one Azure tenant to query an Azure Monitor workspace (Managed Prometheus) in a different Azure tenant, ideally without managing client secrets. I’ve tested a couple of approaches and hit dead ends, so I’m hoping someone can point me in the right direction.

My scenario

  • Tenant A has an Azure Managed Grafana workspace

  • Tenant B has an Azure Monitor workspace with Prometheus metrics

  • I want Tenant A’s Grafana workspace to query Prometheus metrics stored in Tenant B’s Azure Monitor workspace.

What I’ve tried

1. Azure Lighthouse

I deployed a Lighthouse delegation from Tenant B to grant Tenant A’s Grafana system-assigned managed identity access to the Azure Monitor workspace. The delegation deployed successfully with the Monitoring Reader role.

However, querying Prometheus requires the Monitoring Data Reader role, which grants the DataAction Microsoft.Monitor/accounts/data/metrics/read. Lighthouse explicitly does not support roles with DataActions. So Lighthouse cannot grant the permission needed to query the Prometheus data-plane endpoint, regardless of which role you choose.

Notably, Lighthouse does work for Application Insights cross-tenant. I tested this in the same setup: I created a Lighthouse delegation with Monitoring Reader, and the Grafana workspace in Tenant A was able to query Application Insights data in Tenant B via the Azure Monitor plugin using its system-assigned managed identity. The difference is that Application Insights data is accessed via ARM control-plane operations (covered by Monitoring Reader), while Prometheus uses a separate data-plane endpoint that requires a DataAction.

2. Federated identity credentials

I tested FIC two ways: first with a user-assigned managed identity in Tenant B with a federated credential trust to the Grafana workspace’s SAMI in Tenant A, and second with an app registration in Tenant B with the same federated credential trust. The idea in both cases was that Grafana would use its SAMI to get a token, then exchange it for a token scoped to Tenant B via FIC. The Prometheus data source Azure auth docs link to the Entra ID auth docs, which mention FIC support with client_authentication = managed_identity and federated_credential_audience. I haven’t been able to get either approach working for data source plugin authentication though, and my assumption is that the FIC settings only apply to Grafana SSO login. If that’s wrong and there is a way to configure FIC for the Prometheus data source, I’d love to know what I’m missing.

What does work (but isn’t ideal)

App Registration with a client secret in the data tenant works fine. But we’d rather not manage client secrets if there’s an alternative.

My questions

  1. Is there a supported way to query Azure Monitor Prometheus data cross-tenant from Grafana without using a client secret? I’d love to hear if anyone has solved this.

  2. Is cross-tenant support for data source plugins on the roadmap? Adding FIC or a similar secretless cross-tenant mechanism to the Prometheus plugin’s Managed Identity auth path would be very valuable for multi-tenant Azure environments.

  3. Have I missed any other approach? A different plugin configuration, or an Azure-side mechanism I haven’t considered?

Any guidance would be really appreciated. Thanks in advance.

2

Hello

3

Hi @infofcc3 , this is about the Prometheus data source, not Azure Monitor. As mentioned in the post, Azure Monitor already works cross tenant via Lighthouse.

1 Like