How to build a pre-bucketed histogram (with influxdb)

[bojand]

Hello I am using grafana and influxdb. If I already have pre-bucketed histgram data like this:

“histogram”: [
{ “mark”: 0.000265723, “count”: 1, “frequency”: 0.010416666666666666 },
{ “mark”: 0.000785212, “count”: 67, “frequency”: 0.6979166666666666 },
{ “mark”: 0.001304701, “count”: 17, “frequency”: 0.17708333333333334 },
{ “mark”: 0.0018241899999999998, “count”: 5, “frequency”: 0.052083333333333336 },
{ “mark”: 0.0023436789999999996, “count”: 0, “frequency”: 0 },
{ “mark”: 0.0028631679999999993, “count”: 0, “frequency”: 0 },
{ “mark”: 0.0033826569999999994, “count”: 0, “frequency”: 0 },
{ “mark”: 0.0039021459999999996, “count”: 0, “frequency”: 0 },
{ “mark”: 0.004421634999999999, “count”: 1, “frequency”: 0.010416666666666666 },
{ “mark”: 0.004941123999999999, “count”: 2, “frequency”: 0.020833333333333332 },
{ “mark”: 0.005460613, “count”: 3, “frequency”: 0.03125 }
]

mark is the upper bound of the bucket. technically the histogram data is all for 1 point in time.
What’s the best way to insert that into influxdb using line protocol and then query it to display it as a histogram in grafane? I can’t get it to work no matter what I try.

I’ve tried for inserting as:

my_histogram mark=0.000265723,count=1 timestamp
my_histogram mark=0.000785212,count=67 timestamp

if I use the same timestamp, then it seems to only insert one data point into influxdb. I’ve also tried just increasing the timestamp by a ms and that inserts it, but I can’t seem query it properly in grafana for charting.

I’ve even tried inserting things like

my_histogram mark=0.000785212,count=1 timestamp
my_histogram mark=0.000785212,count=1 timestamp + 1ms
my_histogram mark=0.000785212,count=1 timestamp + 2ms
…

where for example this mark would have 67 lines (the total for this bucket) each one being +1 ms. And then querying that so that the count of the same mark would be the count for each line for the bucket, but I couldn’t get the chart properly.

Any help would be appreciated. Thanks!

[johnh]

I, too, have this issue. In my case, I’ve aggregated/down-sampled raw data into 1h intervals to address volume issues. From my research, it seems that there is not a plugin for Grafana that can render this kind of data at this time.

FYI, a solution tothis issue would likely resolve this issue as they appear to be the same issue, so you may want to watch that posting as well (I’m cross posting there).

The scope of this change does not appear to be very large (to someone that knows how to edit/deploy .ts files!). The histogram.ts appears to be where most of this logic would be contained. There would need to be some UI option to identify this pre-aggregated condition as well.

I’ll be interested to know if anyone responds.

[knoguchi]

Untested with Influxdb

Grafana is picky of datasource. A panel may or may not work due to subtle difference of the data structure that the datasource returns. Having said that I was successfully visualized histogram using Bar Chart panel plugin, which is currently beta in Grafana 8.0.3, with SQL db that returns pre-bucketed data as below.

┌────lower─┬────upper─┬──pct─┐
│       0 │   405892 │ 86.7 │
│   405892 │  1238191 │ 12.5 │
│   1238191│  2192080 │    0 │
└─────────┴──────────┴──────┘

I used “upper” as X axis and pct as Y axis for vertical Histogram chart. If the data has one string field and one value field the panel should be able to draw histogram chart without tweaking the config. Bar Chart panel does not require timestamp field at all unlike other panel plugins.

Don’t be confused with Bar Gauge panel. Bar Gauge plugin is old, and requires uncommon data format.

[grant2]

Did you try this using the new Histogram visualization that was introduced in v8? You can find this at play.grafana.org