How to create Prometheus filters

Grafana Configuration
1

I have configured grafana successfully with prometheus database.
I am using grafana v13.0.1 on windows 2019 to monitor windows servers.I want to be guided on how to create functional filters to allow me filter only servers with high resource utilization
GRAFANA FORUM’S 3 BEST PRACTICES:

  1. Please post inside a subcategory
  2. For questions and advice, please try to use the template below
  3. If you post a question and someone solves it, please mark is as SOLVED
    ~ THANK YOU!! the grafana team
    →

  • What Grafana version and what operating system are you using?

  • What are you trying to achieve?

  • How are you trying to achieve it?

  • What happened?

  • What did you expect to happen?

  • Can you copy/paste the configuration(s) that you are having problems with?

  • Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.

  • Did you follow any online instructions? If so, what is the URL?

2

Welcome @ibspace to the community

Verify Your Metric Names
Run this in Prometheus first (http://localhost:9090/query):
promql

windows_cpu_time_total

If no data returned, try wmi_cpu_time_total → use whichever works in all queries below.

Create Two Dashboard Variables
Dashboard Settings → Variables → Add variable

Variable 1 → Server filter:

Field Value
Type Query
Name instance
Label Server
Data source Prometheus
Query type Label values
Label instance
Metric windows_cpu_time_total
Multi-value ON
Include All ON

Variable 2 → Threshold:

Field Value
Type Textbox
Name cpu_threshold
Label High Utilization Above (%)
Default 80

Screenshot (741)
Screenshot (741)1920×906 118 KB

Create a Stat Panel with This Query

Add new panel → choose Stat visualization → paste this query:

CPU:
promql

(100 - (avg by(instance)(rate(windows_cpu_time_total{mode="idle", instance=~"$instance"}[5m])) * 100)) > $cpu_threshold

Memory :
promql

(100 - ((windows_os_physical_memory_free_bytes{instance=~"$instance"} / windows_cs_physical_memory_bytes{instance=~"$instance"}) * 100)) > $cpu_threshold

Disk Usage :
promql

(100 - ((windows_logical_disk_free_bytes{instance=~"$instance", volume!~"HarddiskVolume.*"} / windows_logical_disk_size_bytes{instance=~"$instance"}) * 100)) > $cpu_threshold

Screenshot (743)
Screenshot (743)1920×901 214 KB

Set Color Thresholds

Right panel → Thresholds:

Green Base → 0
Yellow → 60
Red → 80

Color scheme → From thresholds (by value)

Test the Filter

Set cpu_threshold = 80 → only servers above 80% appear
Set cpu_threshold = 99 → only critically loaded servers appear
Panel empty = all servers healthy

Screenshot (744)
Screenshot (744)1920×908 207 KB

The > $cpu_threshold in the query is the key → Prometheus only returns servers exceeding the threshold, others disappear automatically. No extra transformations needed.

Screenshot (745)
Screenshot (745)1920×904 159 KB

3

Hello Shanker,

I am glad to join the grafana community.

I will follow the steps.

Thank you so much for your support.

Best Regards.

4

Thanks @ibspace I just wanted to know did the solution work out for you? I can help further if you’re still running into any issues.

5

Hello Shanker,

Thank you for your support.

I realized that i can always sort the metrics in ascending or descending order which will give me result for ‘Critical’,‘Warning’ and ‘Normal’.
However i will implement your guide in my next project.

Kindly assist me with the request below:

I used the json file attached to create a dashboard for monitoring Windows servers, but i realized that the dashboard captures only C and E drives even though some of the servers have more drives: C,E,F,G.

Please guide me on how to make grafana capture all the available drives, i have confirmed that windows exporter reports the metrics correctly on prometheus using this command windows_logical_disk_size_bytes{instance=“IB_Space:9182”}

Best Regards.

(attachments)

18658_rev1.json (53.4 KB)

6

Hi,

I checked your JSON dashboard and found the issue. The Server Capacity table has hardcoded queries for only C: and D: drives, so drives like E:, F:, and G: are never displayed.

Replace both hardcoded disk queries with this single dynamic query:

100 - ((windows_logical_disk_free_bytes{instance=~"$instance", volume!~"HarddiskVolume.*"} / windows_logical_disk_size_bytes{instance=~"$instance", volume!~"HarddiskVolume.*"}) * 100)

Set the Legend field to:

{{volume}}

If the table still doesn’t display all drives correctly, update the Organize fields and Group By transformations by removing the hardcoded C: and D: column references so the dynamic disk series can be displayed correctly.