Bug? Reduce Row-Transformation not using "Alias", when only 1 input delivers data

[realdognose]

We are currently running on Grafana v11.5.2 (598e0338d5) and noted a possible Bug with the “Reduce-Row” Transformation:

We have 7 queries, returning some solar-production information:

• ac in l1
• ac in l2
• ac in l3
• ac out l1
• ac out l2
• ac out l3
• DC PV

The ReduceRow Transformation is used, to sum up these queries to a single row called Solar.
This generally works fine, EXCEPT a system is only delivering data for ONE of the queries.

In that case the ReduceRow-Transformation does not produce a Row Solar, but Solar DC PV.

While this looked like a feature at first, it causes issues with subsequent transformation using Solar to do some maths. They are all skipped, because there is no Solar DataSet available.

Can the Transformation be forced, to be executed (and produce Solar) even if there is only 1 input-query delivering data? (Else we would probably need to use 6 binary-transformations to sum them up?)

(the transformation does not show the input-queries not delivering data here)

For comparison a system delivering data for at least 2 queries:

[realdognose]

Using single “binary transformations” also didn’t produce the desired result.

So, what we did now:

• Created 2 more queries of a random value, which is guaranteed to be existing. (zero1_query, zero2_query)
• Created a Transformation doing "zero_series = zero1_query - zero2_query
• Added the “zero_seies” as part of the initial reduce-row transformation:

Kind a dirty hack, but delivers the expected result, cause now there are always 2 input queries at least.

[yosiasz]

what is your datasource?

[realdognose]

It’s Influx.

The above workaround does only work to some extend. The underlaying issue is the following:

• When summing up queries through Transformations, grafana gives the result a concatenated name, like joined_q1_q2_q3
• now, if one of these Queries will result in NO DATA, (for example q2) The transformation sums up the remaining queries, but then names it joined_q1_q3
• Any subsequent transformation, that targeted to use the result of joined_q1_q2_q3 now fails, because that reference cannot be found.

So, still seeking for a solution. Basically, what I would need: If a query returns NO DATA - use a time series with all 0`s instead, so the whole transformation execution plan works as intendet despite some initial queries return NO DATA

So, the workaround produces a series called Solar - but it’s internal name will be joined_dcpv_zeroseries rather than joined_dcpv_acin_acout_zeroseries as any additional transformation would expect.

Hence, even if Solar is now displayed - it cannot be used for further transformations reliably.

Query-wise this can’t be resolved as to my research, because all the fill() options can only fill data - but not generate something in case of NO DATA.

[yosiasz]

and are you using flux query language or influxql?

since you are asking for help on existing setup that is not working for you, we will need some sample data of sorts to set things up on our side of things, otherwise it is just swinging in the dark.

[realdognose]

I believe it’s influxql.

The Issue is quite independent of the actual querie(s) and should be reproducable with any kind of query.

But i’ll try to provide some more “real-world-issue-insights”:

To stick with the initial sample data (but it affects all transformations), we have 7 queries like this:
Nothing crazy, just the fetch of a single measurement from it’s autogen table, filling nulls with it’s previous value.

These 7 queries namely are

• acpvoutl1_query
• acpvoutl2_query
• acpvoutl3_query
• acpvinl1_query
• acpvinl2_query
• acpvinl3_query
• dcpv_query

and represent the Solar-Power available on two different AC-connection points (or DC directly)

Thus, to get the total amount of solar, we just use a reduce row as follows:

Now, if we want to do some maths with that value, for example do SolarOverhead = Solar - consumption, the transformation for that would target the joined query of the calculation, which is a wild concatenation of all the queries involved:

This is working - but what is happening now is:
When swithching to another $siteid that returns NO DATA for one of the queries, things start to fail. Here is an example of a site only using DC-PV.

Two things are happening:

1.) First, the queries returning NO DATA are silently dropped from the Solar-Transformation, but that one still returns the result of only DC PV series.
2.) The very long join-query is silently dropped from the Overhead-Transformation - and that one then fails to compute at all:

Looking at the filter-proposals for the Overhead transformation NOW unveils, that the Solar query is no longer referenceable by it’s long joined-name, it just keeps the name dcpv_query as that is the only series having data for the Reduction.

In a case, where there would be two inputs available, it will be shorted from joinByField-q1-q2-q3-q4-q5-q6-q7 to joinByField-q3-q7 which also causes the long reference to be invalid and subsequent transformations to fail.

Example for a site delivering 4/7 inputs:

Here the “name of the solar transformation” is only made out of 4 joined queries, hence the reference to the name with 7 queries also becomes invalid:

---

For the time beeing I now have found a (dirty) workaround:

• We are doing 3 calculations using Solar
• So, I’ve now created 3 * 7 queries and don’t reuse the computation Result of Solar, but rather
  do all 3 mathematical transformations from the (independent) vanilla-queries.

This way, the final name of each transformation “doesn’t matter”, because it is not re-used, but ofc. is not ideal, neither for design-time, nor for performance and resource handling.

Probably a solution (Grafana side) would be to allow to specify a more compact-name for a transformation result - which then doesn’t change, based on the available input-queries.

If I for example could specify that the solar-transformation should always be called joinByFields-SolarResult I could keep referencing that in subsequent transformations, no matter if it is made out of 1 or 7 queries delivering data.

Or, beeing able to do Alias:Solar instead of Query:joinByField... to define the reference.