Zabbix data source Constantly crushing(Error 403 Apache 2.4) - Solved (Apache Mod_Evasive Error)

Grafana
1

Hi Guys!,
I’m using CentOS7, Apache 2.4.39, Zabbix 4.2.3-2.el7 & Grafana 6.2.4-1 installed on my system.
I’ve installed Zabbix Data Source/Plugin and no matter what type of settings/configurations i’ve tried in “Zabbix Data Source”, my graphs are working for not longer than 5min. it showing data, even updating for 5min and then crushing providing me a “Request error”.

When it crushing, i’m running “Query inspector” on either one of the graphs and getting at first 403 error. Once i’m restarting Apache and refreshing the page(Chrome) everything is working smooth for approx 5min and then the 403 errors starting to show up in my web server logs.
Just to mention, i’m using SSL, no worries, i added my keys to the .ini file. seems like working fine.

5min everything works fine, and then graph errors starting to pop individually until eventually the entire dashboard being covered up with “Request Error” per each graph.
Also, in the apache error & access logs i’m getting the following multiple times(per each graph):
“client denied by server configuration: /usr/share/zabbix/api_jsonrpc.php”
“POST /zabbix/api_jsonrpc.php HTTP/1.1” 403 231 “-” “Grafana/6.2.4”

Idk, but it feels like the API connection between Grafana to Zabbix is kinda shaky :).
I’ve attached to this post my Zabbix Data Source configuration for your inspection.
Like i’ve said before, i’ve tried various types of settings at the “Zabbix Data Source” on Grafana, but none of them is lasting for more than 5min until 403 error starting to pop.

Please see below:
Zabbix Data Source

22%20PM
22%20PM665×943 48.9 KB

Any ideas?
Thank you!
Benny.M

–Update–

I was running Grafana from the same server i had zabbix installed on (my web server).
Once i installed grafana on my mac (Remote Client) i’m not expereincing any errors.
+
While i was running it from my web server it was causing all the apps under conf.d/ (e.g zabbix server/PhpMyAdmin) 403 errors as well.

–Update—

Well, after 12hrs testing, even from a client machine i’m still getting the 403 errors from the apache.

HELP :slight_smile:

–Final Update - Solution–

Inside mod_evasive conf file, the “DOSPageCount” needed to be change from 10 to 20(in my case).
When Grafna is connecting to pull data either by server OR browser from Zabbix it’s generating large amount of HTTPS/Server requests per the same page. Therefore it was exceeding the amount allowed by Mod_Evasive security settings in the conf file.
Once i changed it error 403 stop showing up (I tested it for almost a week).
See below:
15%20PM

Enjoy.
Peace out.

1 Like
2

Thank you so much for this absolute gem of information.

I’ve been wrestling with this “403” message for most of today, and after putting my webserver logging to “debug”, I saw the “evasive20:error” message, but the Apache2 mod on Ubuntu is called “evasive.conf”, and there are no annotations to tell you what to change :frowning:

The default “evasive.conf” looks like this:

#DOSHashTableSize 3097 #DOSPageCount 2 #DOSSiteCount 50 #DOSPageInterval 1 #DOSSiteInterval 1 #DOSBlockingPeriod 10 
#DOSEmailNotify      you@yourdomain.com
#DOSSystemCommand    "su - someuser -c '/sbin/... %s ...'"
#DOSLogDir           "/var/log/mod_evasive"

…so I un-hashed the DOSPageCount variable and changed the value to 50 and all my headaches went away.

Thanks again for the pointer :slight_smile:

1 Like