Java + graphite + Grafana -> metrics name issue

Grafana
1

I’m a bit confused with the way metrics from the application are presented in Grafana from Graphite.

I’m using Dropwizard on the client side that sends metrics to Graphite.

My case is - request size monitoring, so metrics is ‘.request…size’. I’m using MetricRegistry#histogram#update to record values

Dropwizard (which is also used internally in Micrometer) sends metrics in GraphiteReporter class, and here’s the code that reports histogram values:

private void reportHistogram(String name, Histogram histogram, long timestamp) throws IOException {
    final Snapshot snapshot = histogram.getSnapshot();
    sendIfEnabled(COUNT, name, histogram.getCount(), timestamp);
    sendIfEnabled(MAX, name, snapshot.getMax(), timestamp);
    sendIfEnabled(MEAN, name, snapshot.getMean(), timestamp);
    sendIfEnabled(MIN, name, snapshot.getMin(), timestamp);
    sendIfEnabled(STDDEV, name, snapshot.getStdDev(), timestamp);
    sendIfEnabled(P50, name, snapshot.getMedian(), timestamp);
    sendIfEnabled(P75, name, snapshot.get75thPercentile(), timestamp);
    sendIfEnabled(P95, name, snapshot.get95thPercentile(), timestamp);
    sendIfEnabled(P98, name, snapshot.get98thPercentile(), timestamp);
    sendIfEnabled(P99, name, snapshot.get99thPercentile(), timestamp);
    sendIfEnabled(P999, name, snapshot.get999thPercentile(), timestamp);
}

I guess, the idea is - instead of sending 1’000’000 numbers in array, it aggregate all calls into ~10 value that give good statistical information.

But as the result, in Grafana I see following metrics:

<node>.request.<endpoint>.size.count
<node>.request.<endpoint>.size.max
<node>.request.<endpoint>.size.mean
...
<node>.request.<endpoint>.size.p999

and it seems strange to me. I thought that all those ‘count’, ‘max’ and percentiles are calculated via Graphite functions and that Graphite should de-aggregate values somehow.

Also I’m very confused when I’m trying to get average numbers for all nodes or all endpoints -> what metric should I use -> .request. .size.avg + some aggregation function?

So I wonder, should it even be like this? Maybe it’s some lack of Graphite configuration (or just my lack of Grafana skills). Because I think that I should trust the was Dropwizard & Micrometer (and thus, Spring) send the data.

This question is also asked on stackoverflow: https://stackoverflow.com/questions/60877388/metrics-monitoring-with-java-graphite-micrometer-dropwizard