Hi All,
I use Grafana to display bug metrics pulled from Jira and store them to Influxdb as data source,
Recently I have started noticing that grafana dashboard started throwing below error when I load my dashboard.
Template variable service failed Failed to fetch
Fail to fetch data on the widgets.
There is no clue of whats happening, is it a grafana issue or Influxdb issue?
Is it a data volume issue or any plugin issue?
When I edit the widgets on dashboard individually and refresh the widget, it shows up data.
But when I do browser refresh, all widgets fail to fetch.
This dashboard use to work before and now it started this behavior, not sure if this due to load, on infrastructure side SRE team has looked into load and it is perfectly fine. Out of ideas
Please help
I am using Grafana 8.5.13
Influxdb 2.4.0
It looks like query used in dashboard variable is failing and then also panel queries are failing. Use browser console and check what’s their responses.
Maybe you have inefficient queries and those are killing your datasource (influx).
I wanted to share one thing which might be causing this issue.
This dashboard is designed to display metrics for various projects selected from the dropdown as shown in the screenshot, that means, all queries are same for all the widgets on the dashboard.
This dashboard was working fine for the selected projects until recently. Even now for other projects, dashboard does display metrics without issues.
One of the root causes - may be a primary one is
Recently there was a mass update to jira bugs for two projects where we are seeing this “fail to fetch issue” as a result there about 800 to 1000 records gets pulled to influxdb.
Seems like the same query which was working for other projects stopped working for these two projects as volume of changes are more in this case.
Reducing the time range didnt help, also tried updating the query but didnt help as this mass update to bugs happened at the same time, so could you please share some thoughts or workarounds how to handle this case. appreciate your help. Thank you
We have a dotnet service which pulls jira data into influxdb to display metrics on grafana dashboard.
One of the requirements is to display Bugs status overperiod(bug trending wrt) of time for example, Development and Escapedefects for a given quarter also for a given year.
Jira gives current status only.
In Grafana our query pulls last status of the jiraid
We are using community edition of Grafana, so is there a limitation on the features?
Lets assume that we need to fix the queries.
If we need to fix the queries used in the widgets, what can be done in this example?.
Query to pull total number of Development defects for a given project.
SELECT last(“value”) FROM “jira_Bug” WHERE (“projectkey” =~ /^$Group$/ AND (“status” != ‘Cancelled’ AND “status” != ‘Done’ AND “bugfoundby” = ‘Development’ AND “jiraid” !~ /^$ClosedDevBugs$/)) AND $timeFilter GROUP BY “Priority”,“jiraid”
Best to read key concepts: tags vs fields which can be indexed and which cannot and design accordingly.
are the following columns fields or tags? projectkey, status, bugfoundby, jiraid
You are also using regex filters !~, ~, which are not performant.
You have historical data, not sure how much, which could affect performance. I would segregate those into another bucket/table or measurement and keep only recent stuff in your main table.
SELECT last("value")
FROM "jira_Bug"
WHERE ("projectkey" =~ /^$Group$/
AND ("status" != 'Cancelled'
AND "status" != 'Done'
AND "bugfoundby" = 'Development'
AND "jiraid" !~ /^$ClosedDevBugs$/)
) AND $timeFilter GROUP BY "Priority","jiraid"
Grafana queries has limitation, that is the only way I know to pull latest record to avoid duplicates and there is transform function to get the count of records.
