Heavy forwarders?

Grafana Loki
1

Hello all,

I’m new to Loki/Grafana, please excuse me in advance if this is a dumb question.
I did some search but I didn’t find what I was looking for, so here I am.

My employer is willing to do a Loki POC, which would require us to use a forwarder.
Basically that would look like:
Logging clients in various on-prem/cloud DCs → Heavy forwarders in our two main colo datacenters → Grafana/Loki cluster in our main Azure subscription
It’s an extra layer of complexity we would use for a few months until a good bunch of our DCs and cloud subscriptions are consolidated/migrated. I can’t wait for the consolidation to happen before starting working on this logging POC.

So anyways, my question is:
Is there a way to use heavy forwarders ? A server that would collect logs from branches clients and forward them to the Loki cluster ?
What exists and where can I read about it ?
I can imagine that would cause issues with the labels, since promtail applies them at the client level ?

Thanks in advance for your help.

2

I don’t understand what is “heavy” in this case. It looks like “receive and send further”. No heavy processing, because that’s should be already done on the client side (labeling, parsing, …)

I would use OpenTelemetry (OTEL) collector as service [Loki receiver/Loki exporter], which will receive (it doesn’t collect) logs from branches clients and forward them to (central) Loki cluster. You can even chain them, so you may have multiple levels of OTEL collectors.

1 Like
3

Maybe I misunderstood the notion of “heavy forwarders” from other logging solutions (splunk).
I’m new to the whole Logging world, not only to grafana.
It appeared to me that since agents are already called “forwarders”, maybe that’s what “heavy forwarders” was. Sorry for the misuse of that terminology.

Anyways. Back to the cool stuff: Thanks a lot for your answer, that’s exactly what I was searching for.
I’ll do some reading on that topic. Thanks !

1 Like