Grafana variable filter is not displaying all variable values

Grafana
1

Hello,

I am using Grafana 10 and influxdb v2.7.4.

I am using below flux code to create grafana variable for my dashboard.
When i test this on influxdb script editor, i can get 3500 Client_Name variable values.
But when i apply this as a variable and open the variable filter on dashboard the values are truncated. In alphabetic order i can only get until variable names starting with f… Nearly %70 of the variable value Client Names are missing in the filter. How can i display all 3500 Client Name variable filter values in the dropdown filter?

import "influxdata/influxdb/schema"
schema.tagValues(
    bucket: "VFBckStrg",
    tag: "Client_Name"
)
2

Hello @grant2 ,

I am getting variables as you suggested before.
There are 3500 values for the tag values.
But when i create grafana dashboard variable with code you suggested, %70 of the values are truncated and not shown in the filter variable dropdown on the dashboard.

3

Below code is displaying 3500 client names in influxdb script editor area.

import "influxdata/influxdb/schema"
schema.tagValues(
    bucket: "Mbuck",
    tag: "Client_Name"
)

But when i create filter variable on grafana with this code, the filter variable dropdown is truncating maybe %70 of values and just displaying 800-1000 client names in dashboard filter variable dropdown.

I also tried below code to create variable with flux on grafana :

from(bucket: "VFBckStrg")
|> range(start: -30d)
|> filter(fn: (r) => r["_measurement"] == "VMwareRawData20")
|> keyValues(keyColumns: ["Client_Name"]) 
|> group() 
|> keep(columns: ["Client_Name"]) 
|> distinct(column: "Client_Name")

This also brings 3500 client names in influxdb script editor. But again it shows less variables in dashboard filter variable. Maybe 1500-1600 of variables. Bu again not 3500 of variables.

I need to be able to choose all 3500 variable value client names in dashboard filter.

4

where are you seeing this “less variables” at? Please share screenshot

5

Hello @yosiasz,

This is my variable :

[Grafana variable creation area]

image
image989×733 60.2 KB

Below is my flux query of dashboard panel :

from(bucket: "Vbuck")
  |> range(start: -30d)
  |> filter(fn: (r) => r["_measurement"] == "VdatData")  
  |> filter(fn: (r) => r["Client_Name"] =~ /^${ClientNameX:regex}$/)
  |> filter(fn: (r) => r["Schedule_Type"] =~ /^${ScheduleTypeX:regex}$/)
  |> filter(fn: (r) => r["Policy_Name"] =~ /^${PolicyNameX:regex}$/)
  |> filter(fn: (r) => r["Site_Name"] =~ /^${SiteNameX:regex}$/)
  |> filter(fn: (r) => r["Policy_Type"] =~ /^${PolicyTypeX:regex}$/)
  |> filter(fn: (r) => r["Owner_Group"] =~ /^${OwnerGroupX:regex}$/)
  |> filter(fn: (r) => r["Group_Manager"] =~ /^${GroupManagerX:regex}$/)
  |> filter(fn: (r) => r["Senior_Manager"] =~ /^${SeniorManagerX:regex}$/)
  |> filter(fn: (r) => r["Job_Status"] =~ /^${JobStatusX:regex}$/)
  |> filter(fn: (r) => r["Service_Name"] =~ /^${ServiceNameX:regex}$/)
  |> filter(fn: (r) => r["Master_Server_Name"] =~ /^${MasterServerNameX:regex}$/)
  |> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value")
  |> keep(columns: ["_time", "Start_Time", "Client_Name", "Policy_Name", "Policy_Type", "End_Time", "Service_Name", "Owner_Group", "Senior_Manager", "Group_Manager", "Schedule_Type", "Site_Name", "Job_Status", "Master_Server_Name", "Schedule_Type", "Job_Status", "Job_Status_Code", "Duration_Minutes", "Written_Size_MB", "Real_Size_MB", "throughput_MB_per_sec"])

Below is variable query in influxdb data explorer script area :

[Influxdb data explorer]
You can see there are lots of client names starting with zbx…

image
image1336×671 68.4 KB

Below is the filter variable dropdown that variable values are truncated.
(Which you asked)

[Grafana Dashboard]
image

As you can see from influxdb data explorer i have lots of client name values starting with zbx…
But i cannot see them in the Client Name filter. I mean i run the query in the garafana variable area in influxdb data explorer. I get 3500 client name tag value results. Lots of the names start with zbx… But when i use same query in grafana variable area and then open the related filter in dashboard most of the client names are not in filter. Then i write name of the Client that i am sure it is in query results. The client names that start with zbx… There should be more than 10 results but it displays nothing.

I know dropdown menu will not show all at once but while searching with keyword in filter area where it says enter variable value. it should show every one of 3500 Client Names by searching.

1 Like
6

ok I see the issue. for testing purposes instead of a Client variable drop down, change it to Variable plugin Variables | Volkov Labs

In case the issue is the drop down. I implemented the same amount of variables and I am seeing them all using the above plugin

image
image426×724 26.5 KB

7

Hello @yosiasz,

My server is in the intranet and does not have internet access to outside. How can i install Variable plugin offline.

Another question. Does activating this plugin affects other dashboards? Because other dashboarda does not have problem.

8

you can read here on how to download the zip file on another machine

Should not affect other dashboards, use it on a test grafana if you have one and you have concerns.

9

Hello @yosiasz,

I installed Variables plugin as you suggested.
It looks handsome.
But still i cannot get all data in filter.

Query inspector show 3524 variable values.

image

image

It displays some of variable values. About 800-900 of 3500 variable values.,

When i search for zbx… it should show over 10 clients starting with zbx… but none.

image

10

Hello @yosiasz,

I also checked chrome browser inspect area.
I see some errors on browser. Is this related.

image
image1298×371 89.6 KB

11

Hi @yosiasz,

There is something strange. I also have influxdb 1.8 version. I have same data in that db too. I tried to get tag values for Client_Name tag from influxdb v1.8 and create grafana variable with influxql, there is no problem. I can see 3500 of client names.

But same data and flux and influxdb 2.7.4 causes problem. Truncates most of the filter variable values.

12

if you type zbx and you do not see anything, could it mean there are hidden value in front of those zbx rows?

....zbx-vader

instead of
zbx-vader

or some other hidden non standard values?

13

Hello @yosiasz,

No hidden values. Let me show you the filter in influxdb 1.8 .

Below is same data but variable is created with influxql like this :

SHOW TAG VALUES FROM “SELECTBOX_VALUES” WITH KEY = “Client_Name”

You can see that zbx- clients are coming. But if i use flux for same data in influxdb v2.7.4 these zbx-… clients are truncated.

The strange thing is.

If i create variable from influx 1.8 with influxql, i can see 3500 client names at variable creation page by preview of values section at bottom of page.

If i create variable from influx v2.7.4, i can only see 1000 client names at variable creation page by preview of values section at bottom of page.

[Variable created from influxdb 1.8 db - Same data]

image
image567×589 40.7 KB

14

We are planning to use Influxdb 2.7.4. But we cannot completely pass to it from influxdb 1.8 version. Because Grafana filter is working with influxdb1.8. But not working with influxdb 2.7.4.
So we cannot shut 1.8 version. This made us dependent to influxdb 1.8 version.
What can we do?

15

Hello @yosiasz,

I think i found solution. But need help.

Normally with flux i cannot get more than 1000 variable values as dashboard variable filter.

But if i define 8000 variable values by creating custom variable on grafana manually, dashboard filter can search all 8000 values and bring in the selectbox.

So i have to use manuel custom variable in grafana.

Now my problem is updating this custom variable in certain time period. Probably everyday.
How can i update custom grafana variable value dynamically everyday.

A script will run and write new custom values seperated by “,” like this : “Server1,Server2,Server3,…,Server7998,Server7999,Server8000”

I need to fill below area with my scripts dynamically.

[Screenshot from Dashboard/Settings/Variables]
image

16

Look at grafana api and you might need to recreate the dashboard via a script on daily basis
Smells fishy, there is just something wrong here.maybe version of grafana?
Cause it works on my pc. Alsl there is definitely errors in console