Looking up imprecise numeric values from Prometheus

Grafana Prometheus
1

  • What Grafana version and what operating system are you using?
    11.1.0, docker

  • What are you trying to achieve?
    I have a couple prometheus metrics generated by a weather station that i want to use to look up values from psychrometric charts. This is tricky because
    a) Psychrometic values generally do not have equations to calculate them, they have been experimentally determined and are available as n-dimensional charts
    b) there are no psychrometic charts as standard databases, the closest things are libraries which take input values and return an output value.
    c) even if there were a psychrometric table, the metrics in prometheus are real-world data and so continuous, so i cant just join prometheus to a pre-built table using transformations (unless there is a way to fuzzy join?).

  • How are you trying to achieve it?
    My hope was that i could create an http endpoint and use a grafana transformation to lookup each value in the time series via a rest call, but that doesnt seem to be a feature (yes it would be slow, but at least it would work).

As far as i can tell there also isnt a prometheus feature that would allow this kind of lookup.

As an example, i have two metrics:
TDryBulb {}
TDewPoint {}

I want to get and graph the relative humidity from those two values, which if i were doing programmatically i could do with PsychroLib API Documentation — PsychroLib documentation.

Thanks!

2

Hi,

If you want to go with the api approach, gou can install Infinity Plugin - that would allow you to query any REST API and parse JSON / CSV and some other types of common formats.

Can you provide some more info on the data? Ideally you could provide some kind of example series from your prometheus instance and your desired output.

3

I looked into the infinity plugin, but it has the problem still that it returns a table, which i would have to join against (at least to my understanding).

So, some example data could be:

timestamp, dry_bulb, dew_point
12345678, 43.2, 60.5
12345679, 44.2, 60.6
12345680, 43.9, 60.7
12345681, 43.6, 60.1
12345682, 43.7, 60.2
12345683, 43.0, 60.5

(keeping in mind its prometheus, so one series for each non-timestamp column)

What i want is to add a derived column relative_humidity, which if this were sql (and there was a macro to generate the value) I might generate with:
select ts, dry_bulb, dew_point, relative_humidity(dry_bulb, dew_point) from input

However, since this is prometheus, I am relying on grafana transformations to derive the column. In theory i could join the input data against a table which has all possible values of dry_bulb and dew_point with the corresponding relative humidity, but that would require a very large lookup table in order to have every possible number up to a number of decimals. I could generate that as a csv which the infinity plugin pulls, but it would be very large.

4

What I’m missing is can you calculate relative_humidity based on only the current values of the points? If yes (e.g. using some arithmetic operations) you can calculate that in Prometheus. If not (e.g. you need some other data stored in different timestamps) we would have to get… more creative and I don’t know if it’s possible.

5

So what is this formula in plain math

Relative humidity = (specific humidity) / (saturation point) x 100

?

6

That’s the fun part: there is no plain math. This really threw me for a loop, but there is no equation that you can put values in and get the result out. The only way to get the value is to look at a table that someone experimentally created and find the closest value. https://www.psychrosim.com/ as an example. (for the record, i am not a physicist and do not understand why this is the case)

So, i could write a function relative_humidity which takes those values and looks them up in the table and returns the closest one, but prometheus doesn’t seem to support that natively.

7

Why did you opt for Prometheus as backend?

Can you ingest that data into Prometheus also?

Maybe join the 2 data points via a common field

Otherwise…

8

Prometheus made sense for the data collection aspects, as an easy way to retrieve data from large numbers of micro controllers without ddosing a conventional transactional db.

I dont think i can ingest it, because it isnt time-series, and is essentially up to a 5 dimensional array in shape.

My backup idea is to send the data from prometheus to another db, but that isnt a clean solution.

1 Like
9

Might want to look at business text plugin and leverage JavaScript to do what you want to do from 2 data sources.

Are you comfortable with JavaScript and which visualization are you aiming for

Also without real world sample data, it’s hard to try to help you

10

That could work, but at that point i might rather just export the prometheus data to another db which i can do things more normally in.

Alternatively, add an intermediary prometheus passthrough service which generates the values as they come through.

I provided some sample data above, but if you really want “real” data:

timestamp,bus_voltage,humidity_ambient,irradiance,load_current,load_power,load_voltage,pressure_ambient,shunt_voltage,temperature_ambient,temperature_t1,temperature_t2,temperature_t3,temperature_t4,temperature_t5,wind_heading,wind_speed
1729461652.24,2.847,50.188,1000.861,5.0821,106.348022,20.926001,100.608,1.306,25.588,15.064,16.901,17.549,18.524,19,15.313,15.796
1729461682.24,2.004,50.64,1000.013,5.0053,104.475624,20.872999,100.238,1.869,25.296,15.588,16.42,17.718,18.287,19.205,15.785,15.418
1729461712.24,2.079,50.948,1000.267,5.0571,103.200233,20.407,100.634,1.887,25.999,15.121,16.32,17.943,18.533,19.266,15.933,15.225
1729461742.24,2.574,50.216,1000.584,5.0959,105.266006,20.657,100.005,1.255,25.505,15.765,16.8,17.376,18.632,19.024,15.432,15.7
1 Like