Hi everyone, I’m playing with the new Grafana Cloud feature: Adaptive Metrics, but I’m having some issues when using the CLI.
I asked to get the permissions to use the Adaptive metrics for my organization to test it, I received it but when I run this command:
./adaptive-cli.darwin.arm64 --user $USER --url $URL --password $PASSWORD show recommendations --verbose
I got null and no data is showing. I don’t know if there is something that I’m doing wrong of if this feature is not fully working on Grafana.
Hope you can help me with this.
Regards, Felipe.
Hi there –
There are a few reasons you might not have any recommended metrics to aggregate:
-
You hadn’t accessed your account in a while: While we normally generate recommendations every 24 hours, if your account had seen a sustained period of inactivity, we won’t generate recommendations. If this was the case, we would expect that you’d see recommendations within the next day or so (since it sounds like you’ve since logged into your hosted Grafana).
-
[more likely] We weren’t able to identify any metrics that we think we can aggregate without breaking your existing dashboards or alerting/recording rules, or queries run in the last 30 days. Adaptive Metrics will ONLY recommend aggregations that it can guarantee will respect all existing usages of a metric, so if it can’t do that, you’ll have 0 recommendations.
Why might #2 happen?
- Sometimes we see users create custom dashboards or recording rules to understand their metrics cardinality. These often count every single metric in their environment; these uses can result in a metric being marked as off-limits for aggregation (again because we can’t aggregate it without breaking that count query). This could also happen if you’re running “mimirtool analyze”.
- You’re not sending us the necessary metadata to make recommendations. Adaptive Metrics uses this metadata to suggest aggregations that are mathematically correct given a metric’s type (e.g., it doesn’t make sense to compute an average of a counter). See here for more detail.
If none of these seem like they’re relevant for you, please file a support ticket with us and we’ll take a closer look at your specific environment!
2 Likes
Hi Jenvilla, thanks.
In Grafana, we have Cardinality dashboards, but I am unsure if they are being used by anyone. In order to utilize and ensure the proper functioning of the Adaptive Metrics service, we would need to remove those dashboards and wait for 30 days before re-running the queries. Or how should we proceed in this matter?
Best, Felipe.
Hi Felipe -
Since the Cardinality Management dashboards are built by us, we have built the recommendations service to ignore any metrics accesses from those dashboards. So any views of the cardinality management dashboards should not impact the recommendations you get, which means those views would not explain why you have 0 recommendations.
Do you have unused metrics when you look at your cardinality management overview dashboard?
1 Like
Thanks for the explanation, I checked the dashboards and I saw unused metrics.
So I still don’t know why I got null when I made the queries.
Also, I checked that I’m using the same endpoint URL and user but still getting Null.
Another detail is that when I make the queries using curl I get as the response:
[]%
I don’t know if this is an important detail that can make us realize some kind of error.
