Flux query to list contents of a field to be used for a variable

Grafana InfluxDB
, ,
1
  • What Grafana version and what operating system are you using?
    Grafana 9.1.3 on Oracle Entrerprise Linux 8
  • What are you trying to achieve?
    I am trying to select data in a specific field to be used in a variable for a dashboard
  • How are you trying to achieve it?
    Using Flux
  • What happened?
    No data is displayed
  • What did you expect to happen?
    Data to be displayed in the selected field
  • Can you copy/paste the configuration(s) that you are having problems with?
    yes
  • Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.
    no errors. Is is not displaying any data.
  • Did you follow any online instructions? If so, what is the URL?
    I have tried numerous websites.

I have a bucket that receives data from about 2500 hosts. Our hostnames contains a group name. An example would be:
host-webservers-[aws_instance_id].domain.com
host-mysqlservers-[aws_instance_id].domain.com
host-reverseproxy-[aws_instance_id].domain.com
host-postgresql[aws_instance_id].domain.com

We have multiple of each of these types of hosts to the total of about 2500 hosts.

We use telegraf to collect and send data to influxdb.

I have set up a dashboard with various telemetry and other data and am using 3 variables to get select which hosts to look at.

The 1st variable is called bucket as we have various buckets for various AWS regions. So a bucket name will be something like us-east-1.

The second variable I am using is called e_group_name. This varialbe is supposed to list all hostnames and then extract the bit after host- and before the -[aws_instance_id].domain.com. I use this query for that:
import “influxdata/influxdb/schema”
schema.tagValues(
bucket: “${BUCKET}”,
tag: “host”,
predicate: (r) => true,
start: v.timeRangeStart,
stop:v.timeRangeStop
)

selecting the middle bit with this regex:
/.-e-(.)-.*/

This somewhat works. I have found that if i run this query on the CLI using influx query, the host names looks like this:
host-webservers-0deb191d110a7aef0.domain.com
host-webservers-0df933d7bec171e77.domain.com
host-webservers-0e4084661de823ad6.domain.com
host-mysqlservers-0ea0e00928a3a637f.domain.com
host-mysqlservers-0f1fc1e7faa79467e.domain.com
host-webservers-071904f6d18c00ef5.domain…
host-webservers-0a0a59429c1854bfe.domain…
host-mysqlservers-0c4ea0357f553388c.domain…

The query stops listing any further data after the 1st ocurrence of a host that contains the 3 dots. So my query only returns the 1st 5 records in thois sample query.

The third variable is called e_name and uses the following:
import “influxdata/influxdb/schema”
schema.tagValues(
bucket: “${BUCKET}”,
tag: “host”,
predicate: (r) => true,
start: v.timeRangeStart,
stop:v.timeRangeStop
)

selecting the middle bit with this regex:
/csm-e-${e_group_name}.*/

This then further impacts on the 3rd variable that is supposed to return the full hostname, but because of the 3 dots, it is not displaying those hosts and as with the first query, only the 1st 5 records.

I have now been struggling with this for more than 2 weeks and I just cant get the flux query to give me the data i need for the variables as specified above.

Is someone please able to help me with this?

Thank you

2

Aplogies, I meant to say that I am able to see all the data in the influxdb UI.

3

if i understand well when your passing through your second $var it’s value is “some_hostname…”
Instead of full value “some_hostname_1_x_y”?
Would you provide a screen of this behavior ?
you can visualise $var in your query inspector in grafana.
You can check this link maybe if format is different :

4

The same happens on the 1st var. The 1st var is supposed to extract the type of server from the host fqdn and the 2nd var is then used to group the hosts according to the result from the 1st var.

5

But the three little point are just visual effect isn’t it ? where this happen ?
in what do you need to display that ?
did you try a table a fix his colummn width eventualy ? can you share a screen of it , it could be faster …

6

I think I am really confused as in the query inspector I see all of the data as I want to see it, but when I use the same query in the variable query it display just a few options.

7

This is what I see when i run influx query on the cmd line and I suspect os happening in the var query in grafana:

This is what I am expecting to see in the var query but am not but I now see in the query inspcetor.
host-webservers-0deb191d110a7aef0.domain.com
host-webservers-0df933d7bec171e77.domain.com
host-webservers-0e4084661de823ad6.domain.com
host-mysqlservers-0ea0e00928a3a637f.domain.com
host-mysqlservers-0f1fc1e7faa79467e.domain.com

8

The 1st var has this as a regex to select the relevant records:

/.*-(.*)-.*/)

this means that the 1st var should return the following options:
webservers
mysqlservers

The second var has this as the regex
/host-${var1}.*/

which should return:
the fqdn of the servers the var1 relates to.

9

you said you see the correct result in you query inspector. then it’s working as intended.
how do you need to display that ? Do you use a panel ?

10

try to display a basic query in a panel to see how it’s work.
after this use query inspector to use your var in the query panel, and use his value in your query.

11

I use this in a dashboard to display various panels related to the servers. It is only returning a small subset f data and I cant see why that is.

12

I suppose my question would then rather be: why is grafana not returning the full set of data that I need?

13

what do you mean by it’s return a small subset ?
the three little dot ?

man you are avoid my question, don’t acknoledge any of my advice, don’t share screen
How am i suppose to help you

14

could this be related to your regex ? is that working with a simple var ?

15

so, when i run this:

image
image947Ă—186 16.9 KB

i get:
image

What I am expecting is something like this:
image

For the frist var is am expecting to see this:
image

A complete list should be 100 options.

16

In grafana though i see this:

image
image545Ă—857 79.8 KB

In my dashboard i get this for the 1st var:

image
image1222Ă—807 60.6 KB

and this for the 2nd:

image
image896Ă—842 61.9 KB

So the 2nd var will work if the 1st var returns all of the values, but it does not.

17

OK now we can see everything is working query influx ok,
and both var are not empty. I can conclude this a query or regex issue since you don’t have the correct value.

btw
i see your query result in linux console
image
this is not related to influxd, not realted to grafana… this is related to your console width, correct value are indeed here but it’s display is truncated

Why not use influxdb GUI to run query at your http://ip_influxdb_host:8086
This will make your life easier !

18

to be able find you query/regex issue you can create new panel on dash board for each variable to display $var value when you select it and compare to your attended result, you will find out if there something not expected.
In worse case manually replace var value in a new query, run it independantly in influxdb
(use query inspector ofc=

19

Thank you for your help, but it does not matter what I do, the regex seems to be correctI Am just unable to get the correct results in grafana.

20

the thing is grafana is sending to database exactly the string in your query inspector.
the query result in grafana and the query result in influxd can’t be different this have no sense.

maybe you are querying the wrong database host or wrong bucket and it very similar and you don’t find out yet ? something like that already happen here

1 Like