Open source github project URL JaaduVision
In order to reuse existing proxy path and reduce the work on firewalls, iptables, developed agentless solution - python scripts running on host being monitored post stats and logs to central web server which can optionally have Prometheus / influx dB locally or on remote hosts.
OS performance gathering is done using sar; if already enabled on host being monitored, python psutil or custom logic. Performance collection supports custom configuration in CPU, memory, swap, disk/file system, network, and process level CPU/Memory utilization.
Application performance gathering is done by scraping the log files using custom spec for each service or functionality type. Data to be gathered can be specified via regular expressions with pass, fail, count, average, sum, delta type of computation. Log lines of interest can be sent to Loki for visualization along with performance stats. Stats and log collection supports overload control - runs at lowest priority, suspends processing of priority 3, 2, 1 level events at different CPU threshold levels, sends desired number of log lines per collection interval (not all log lines), along with total count of log lines seen.
Application stats can be sent to Prometheus for real time tracking or InfluxDB where guaranteed collection is needed. Python script running on host being monitored saves the stats locally on monitored host when central web server is not accessible, pushes the data later when central server is available. Local caching and retransmit duration can be customized.
This solution works on Linux (Redhat 6.8+, Ubuntu, Debian, Raspberry Pi) and Microsoft Windows with python 2.6+.