queryError: expanding series: closed SeriesSet

  • What Grafana version and what operating system are you using?

  • What are you trying to achieve?
    I simply want to read the data from my bright computing clusters.

  • How are you trying to achieve it?
    Bright supports prometheus out of the box.

  • What happened?
    queryError: expanding series: closed SeriesSet
    Thing are inconsistent. If I search the query in the explore tab searching a specific node it works just fine. As soon as I put it into a dashboard everything starts throwing errors.

  • What did you expect to happen?
    Data to be shown in the dashboard.

  • Can you copy/paste the configuration(s) that you are having problems with?
    gpu_utilization{hostname=~“$node”, parameter=“average”}

  • Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.
    Status: 500. Message: queryError: expanding series: closed SeriesSet

  • Did you follow any online instructions? If so, what is the URL?
    Bright computing has a support doc for adding prometheus to grafana.
    How do I use Grafana to visualize monitoring data from a Bright cluster? | Bright Cluster Manager Knowledge Base

I think I figured it out. My variable was querying too much and taking too long and it was timing everything else out.