Reducing loading time for time series panel with multiple PromQL queries

Context:

  • What Grafana version and what operating system are you using?
    Grafana v8.5.4

  • What are you trying to achieve?
    Minimize loading time of dashboard

  • How are you trying to achieve it?
    Studying the different queries that are executed for the load

  • What happened?
    I am building a time series panel to visualize the traffic that goes through an application. For that I need to see three things:

  1. Total traffic (line in white at the top in the screenshot below)
  2. Break down of that traffic by each component of the application (stacked view of the different lines in the screenshot)
  3. the week on week trend of the total traffic (the white dotted line on the screenshot)

I want to see all three elements graphed in one panel. So I created a time series panel with three queries, each one fetching the data corresponding to one of the points above, and I do get the graph I am wanted:

  • What is the problem?

The loading time for this panel is too long. Mind you, I am using recording rules:

When I analyze the loading time of the panel, I see that the query takes ~17s:

In an effort to understand why, I created three new time series panels, each one graphing only one of the three points mentioned above. As you can see in the screenshot below, the longest one only takes about 6 seconds to load:

This leads me to believe that in the original panel, the queries are executed in sequence (otherwise, the total query time would have exceeded the max of the three, i.e 6.3s)

Question

Any idea if I’m right or wrong? Is there a way to force parallelize the execution or any other lead to reduce the total query time?

Thank you!