How to create alert based on multiple queries?

For few days now I’m trying to create alert when ratio produced to consumed messages is not equal to 1. I don’t want to do it based on prometheus due to each time service is restarted, counter is reseted also I don’t want to create gauge because I’d have to store count of produced messages somewhere in db. I decided to use logs stored in elasticsearch. Here the problem is that service which produces messages is logging to one elasticsearch and service which consumes them to another elastic. According to my knowledge it is impossible to create alret based on 2 queries.

I’m out of ideas how to approach this in other way around. Maybe I’m missing something or there is other way how to create alert like this based on 2 data sources?

You can use --Mixed-- data source then you can choose elasticsearch data source that you need, before it, build two elasticsearch data source for each index.
You can create multiple metric using mixed data source.

Other way,
First, choose the metricbeat, for example, then build the metric,
Second change the data source for heartbeat, for instance, then Add Query, choose the suitable datasource, then you can show in the single graph, multiple datasource.

Then, you can set the alert for each metric with specific conditions…

Regards,
Fadjar Tandabawana

I have the same issue but I cannot really undestand @fadjar340 reply. Could you be more specific? I cannot follow your train of thought.

Sorry for my bad English, not my mother tongue…

Oke, I’ll explain

  1. Create the different datasource, in my case, I use metricbeat and auditbeat
  2. Build new dashboard using Graph Visualization
  3. Select the datasource to --Mixed–
    image
  4. Add first query A for first datasource by selecting the datasource using +Query, select Metricbeat
    image
  5. Build the query as you need…
  6. Add second query B for datasource auditbeat using select the datasource using +Query

    Above steps, create multiple query in the same Graph

Now, we go to the Alerts
7. Open the Alert tab and create the alert using the Create Alert Button
8. Create the Alert for Query A


9. Add the alert for query B using sign +
10.You can select the condition of the alerts, OR - AND

11. Change the second alert to query B

12. Now you have multiple alerts in the same Graph with multiple datasource.

13. SAve the dashbord first.
14. You can test the alert using Test Rule button in the last of the configuration
15. Below the test result

I wish this helps you…

Thanks a lot for your detailed reply. Unfortunately, this does not help me since I want to do math between these two queries and not just check boolean conditions. I sincerely appreciate your reply though!