We are trying to figure out if we can replace Thanos with Mimir in our environment. For that we have deployed both side by side and comparing different dashboards to see if they are showing same data…
Currently we have deployed following in our test environment:
- Grafana Agent Operator + Thanos (currently in use)
- Grafana Agent Flow + Mimir (trying to replace no 1 with this)
As part of evaluating Mimir, we came through following query that is used in one of the dashboards. The results are different with Thanos and Mimir (not only for this query but all the similar queries where we are expecting both chart (range query) and tabular (instant query) results
The query we are using is:
I am running this query for data of last 1 hour (though the result is same for any duration). The Query expects results for Range and Instant queries. The problem is, Thanos returns results for both range and instant query but Mimir returns results only for range query, not for instant query.
Update: I have noticed that this behavior is only when recording rules are used in the Query. If I use raw query, it works as expected.
Thanos shows both Time Series chart/graph (range query) and Table data (instant query) with latest aggregated values as seen in following image:
Where as Mimir shows Time Series chart/graph (range query) but does not show latest aggregated data (instant query) in tabular format as shown in following image:
Update: I have noticed that this behavior is only when recording rules are used in the Query. If I use raw query, it works as expected.
But if I run the same query for now-1h to now-1m (latest data before 1min) Mimir shows both chart (range query) and tabular (instant query) results as seen below:
Update: I have noticed that this behavior is only when recording rules are used in the Query. If I use raw query, it works as expected.
Why Mimir show aggregated table data for duration before 1 min and not for current(now) time? How can I make Mimir show both aggregated chart (result of range query) and table data (result for instant query) for current time?