gRPC OpenCensus monitoring

Grafana Prometheus
1

I’m trying to setup a dashboard for our C++ services running gRPC, but I’m struggling with the latency metrics.

I have grpc_io_client_roundtrip_latency_cumulative_ms_bucket, now this has buckets defined in the labels (or whatever you call it): grpc_io_client_roundtrip_latency_cumulative_ms_bucket{...,le="100000.000000"}

How do I turn this into a heatmap in grafana?
Also looking for any other tips on gRPC monitoring beyond just this.

2

I have been looking at https://github.com/grafana/grafana/issues/10009, but I have to admit that I don’t understand it.

3

Solution in the linked issue.

I have created a bug report. This really looks like a bug to me. The documentation seems to be suggesting that this should work as it is.

4

Ok, after a lot of trial and error I managed to resolve this, but the documentation for Heatmaps is absolutely useless.