Elasticsearch stack monitoring with Grafana v8

Hello!

I’m currently using Grafana to perform multi-cluster elasticsearch stack monitoring. I’m using a modified version of this out-of-date dashboard: Elasticsearch Monitoring based on X-Pack stats | Grafana Labs

A friend sent me this link a while back: How we’re working with the Elastic team to make the Elasticsearch data source for Grafana even more powerful | Grafana Labs and this piqued my interest:

Up to now, Grafana was aiming to support only OSS-Elasticsearch features, but starting with Grafana 8.1.0 (in beta now), the target has shifted to include X-Pack features that are still free to use.

I was wondering, is anyone using grafana for elasticsearch stack monitoring?
If so, how are they doing it?
Do the latest updates from grafana in partnership with elastic make this any easier?

I’ve upgraded my grafana to v8.2.5 and I can see that there is now an XPack Enabled switch on the elasticsearch datasources, but it’s not entirely clear what this actually does…

Thanks. So it’s actually including a couple of XPack features, but not the ones I want. :sob:
Ah well, I live in wait.

Hi @nathan3547 I also want to monitor Elasticsearch, I would be grateful if you could share your experience with me.

Well, there is a good dashboard for monitoring most features of elasticsearch - disk space, search speeds, index speeds etc, which is great. It’s a little bit out of date, but doesn’t take a lot of work to fix.

BUT if you want to replicate elastic stack monitoring (list nodes and their availability, see shards moving etc) then you can’t do this in grafana - kibana is still the best way.
If, like me, you wanted to do multi-cluster x-pack style monitoring without having to pay for the gold/platinum licence (in my case, multi-cluster monitoring was the only thing I needed from that licence tier, and it wasn’t worth the money), then you can use this: GitHub - ndtreviv/elastic-stack-monitoring: Kibana-like multi-cluster monitoring for elasticsearch which I wrote for that purpose.

If you do end up using elastic-stack-monitoring let me know how you get on!

1 Like

Thank you very much for answering me! Honestly, I am a beginner in Elasticsearch and Grafana, I want to monitor Elasticsearch which has only one cluster and one node so that I don’t have to check the data that is inserted into our Elasticsearch indices every day directly in Elasticsearch itself. I want to have a dashboard in Grafana For this, should I install the exporter on the server where Elasticsearch is installed and then do it through Prometheus?
Or I can do this without installing the exporter and without using Prometheus, using X-Pack?
I would be grateful if you could share any link or document that would help me improve in my new work.

If you’re only using a single node it sounds like you don’t have a lot of data and have a generally slim lined tech stack. If it were me, I’d enable X-pack stack monitoring and use kibana. There’s no need to bring in prometheus and grafana.

There are plenty of docs over at elastic, as well as a community much like this one at https://discuss.elastic.co/ not to mention tons of resources on the web to help you out.

1 Like

Thanks, so what do you suggest I activate x-pack in elastic? How does x-pack work? Does it add an index of metrics in elastic that I can add to grafana as a datasource? This is not a personal project and Grafana is used in the office and I cannot use Kibana

This should help: Elasticsearch Monitoring based on X-Pack stats | Grafana Labs

1 Like

Hi, I’m trying to monitor my elastic cluster with that dashboard, but i can’t retrieve any data.
How did you fix that dasboard to show data from elastic 8.X?

No, I ended up abandoning.
What I actually wanted was multi cluster monitoring without having to upgrade to a massive elastic licence - I literally didn’t want any other licence benefits from the potential upgrade, just the multi-cluster monitoring.

As a result, I wrote this: GitHub - ndtreviv/elastic-stack-monitoring: Kibana-like multi-cluster monitoring for elasticsearch

We don’t use grafana anymore - just Kibana and my ESM component.