Grafana for processing monitoring

Just wanted to start this thread as we are using Grafana v4 in Yandex.Money to monitor operations in real time.
For that we created Oracle packages to collect data and send it (by Oracle scheduler) to Grafana server directly.
And i would say it works perfectly!

However all our dashboards are quite identical and it would be great just to enhance the view of some of them.

So i prefer to ask step by step about how to change some features in that thread…

So the first question is:
We have a graph to see the activity (“today so far” timeframe)
Is there any option to add day-by-day trendline based on the data from previous period? (previous day the same time)

Second question:
We have a data based on the country/city (iso code/just text) of purchase.
Can we use some plugin to show the operation frequency on the world map?
Or, for example, based on the IP address of the requestor…

1st question:

When you say

Do you mean you have written your own data source or are you saving it in a time series database? Time shifting can be done for a whole panel using Time Range Overrides or per query by using functions from a time series db (for example, Graphite has the timeShift function).

Question 2:

Yes, to both. With the Worldmap panel, you can either match by country code or by using Geohashes. Elasticsearch has built-in support for geohashes which can be used together with the Worldmap panel. But it is possible to use the Worldmap panel with any data source that returns table data with the geohashes precalculated.

Here is an example using InfluxDB matching on Country Code:

Thanks Daniel,
TimeShift works perfectly!

Regarding worldmap panel, I will ask admins to install it and will come back with the questions…

Regarding data storage, we save data in the default Grafana database. We did not create new custom storages…

Regarding Worldmap Panel
Could you please assist to configure dashboard for the following already existing metrics?

Looks like you are using Graphite.

For Worldmap, you will have to change the queries slightly. The data that is included in Worldmap uses two-letter codes for countries. If you only need support for 9 countries, then you could hard code that using the alias function. For example, alias(UK) would be alias(GB), alias(HongKong) would be alias(HK), alias(Belarus) would be alias(BY) etc.

On the Worldmap tab, then you would choose countries:

If you need support for more countries, then I could add the three-letter codes for countries to Worldmap.

3 code countries would be great as this is a standart for MasterCard auths (https://en.wikipedia.org/wiki/ISO_8583)
However 2 code countriex are also there for the US states. LA - Los Angeles (USA country)… That would be perfect to add it aswell…

Also for Visa cards 2 code countries are used… But it is not an our case for now.

Worldmap has US States already. Just choose states instead of countries for Location Data.

Ok, will add 3 country codes.

Thanks, Daniel, could you please post the notification here for us to update Worldmap plugin on our side accordingly?

Also could you please help to clarify how to display map with the datasource of IP addresses? There are some free IP address databases where IP could be linked to the Lat/Lng coordinates based on the link where it is connected to.
Is there any chance to use such databases and link them using some plugin to display the stats?

Here is an example of IPGeo free database http://dev.maxmind.com/geoip/legacy/geolite/#IP_Geolocation
No doubts, that would be very usable tool.

I updated the Worldmap plugin (version 0.0.17) with country data with 3-letter codes.

Here is an example of using the maxmind db with logstash and ip addresses with InfluxDB:

https://github.com/grafana/worldmap-panel/issues/30#issuecomment-236913993

This example saves them as tags in InfluxDB so that won’t work with Graphite. Not really sure how you would do it with Graphite. Do you have ElasticSearch or something similar that you could use for the IP addresses?

Daniel, obviously It is very fast change you did. Thanks.

So now i used this plugin and it works perfectly!!! Thanks.

1 Like

Daniel,
It looks like it shows something strange :slight_smile:
I see the same data on the graph and for example for UK it shows 1 mln operations, but on the map it shows 4, why and how to correct?

What aggregation are you using? Sounds you like you want current rather than total?

There is no visual difference between current and total aggregation.
For example for “today so far” for USA, Map shows = 4
But real cumulative data is = 2067

Or Map shows %?

No, it doesn’t show percentage. It’s been a while since I looked at this but the details are coming back to me now.

When using Graphite with Worldmap the default for Max Data points is 1 as you only want to have one data point per country. With Graphite, this will by default, consolidate by average. To change the consolidation, use the consolidateBy function like so:

Or just remove the 1 from the Max Data Point field and use the consolidation functions in Map Data Options (though depending on the timerange and amount of data points, this will be usually less accurate due to Graphite consolidation).

Going to go and document this before I forget the details again. Thanks for pointing this out!

Thanks again! Now it looks much more realistic.
Another question: most biggest part of operations coming from RUS :slight_smile: so it would be more interesting to exclude this 1 country from list… Is that possible?

Also is there any chance to use gradient colors instead of predefined/configurable 3?

Regarding the map itself and country coordinates. Could you please check those 2? (see image)
Also i could not see the details when i move cursor to the point. Sometimes it works but mostly not…

  • You can use the exclude function in Graphite.

  • I had a PR for more flexible threshold colors that didn’t get merged so you are stuck with 3 for the moment (same as the SingleStat panel).

  • It is fully possible that latitude and longitude for one or two countries are a bit off. It’s based on a data set from the Worldbank but it’s not very scientific how they figure out where the center point should go. I’ll compare Canada and Sri Lanka to the 2-letter country data and fix them.

  • Do you think you could show me a gif of the mouse overs not working? It seems fine to me - I’m on Chrome and Ubuntu. What browser are you using?

I am also on Chrome.
Here it is

1 Like

Hi Daniel,

Just wanted to mention that it would be great to make legend clickable for the WorldPanel.
For example to see most biggest areas…

Also i wanted to ask your recommendation about how to show the table of data like this (just an example):
Card Product - Auth Type - Count
Plastic - PayPass - 100
Plastic - E-Com - 50
ApplePay - PayPass - 200
ApplePay - E-Com - 120

For the chosen period of time… Because as of now, we are using 2 diggerent graphs for that. Plastic - Count/AuthType - Count… with only time correlation…

Also we have a finance dashboard where we show the pairs of graphs like this:
Purchase - Count / Purchase - Amount
MoneySend - Count - MoneySend Amount
Any ideas on how to merge pairs (amount/count) based on the metric type into 1?

And the last but not the least.
When i choose the today so far timeframe, graph panel shows correct data. :slight_smile:
But when i choose for example 7 or 30 days it shows much less than it is really in the database, why?