Multi-server monitoring setup - looking for best approach

Grafana Configuration
1

Hi all,

I’m setting up a monitoring dashboard for several services I am using and could use some advice.

My setup

  • Main server: has internet but isn’t publicly reachable, where I am hosting Grafana container (Grafana v12.0.2).

  • EC2 instances: running Apache, Docker/Laravel apps

  • C++ application which produces text logs

The problem is that the remote services cannot directly push the information eventually to a Loki/Prometheus service on my server, so I need something in between to collect them.

What I want to monitor

  1. Logs from a C++ app (currently just plain text files).

  2. Apache access/error logs.

  3. Uptime checks for web services (Apache, Laravel application, etc.).

My idea

Install Grafana Alloy (or Loki+Promtail but I have seen that soon is going to be deprecated and here is where my confusion is beginning) on each server to collect logs/metrics and send them somewhere.

Questions

  1. Is Grafana Alloy the right tool here, or should I look at Promtail or anything else?

  2. Best way to ingest plain text C++ logs into Loki?

  3. Does Alloy handle Apache log parsing well, or will I need heavy regex work?

  4. Should uptime monitoring also go through Alloy, or use something like Blackbox Exporter?

  5. Any performance issues with running Agent on lots of servers?

I could use Cloudwatch Logs but I would like to avoid additional costs because of that.

Has anyone here built a similar setup? What worked well, and what would you avoid?

Thanks!