Configuring multiple query-frontend in non containarized environment

i am running grafna loki in a microservice model however the deployment is not containarized … instead i am using indivisual VMs and F5 load balancer
currently i am running
4x ingester
2x qurier
1x query-frontend
3x distributor
we did all sorts of load test everything is working as expected

now i want to add 1 more query frontend
problem is unlike schedular, frontend does not have ring based discovery
so i have to depend on load balancer
i configured
this load balancer resolves to
frontend1: <grpc_service_port>
frontend2: <grpc_service_port>

now problem is i think this f5 balancer (tcp protocol) works fine for http but when it comes to grpc … since its terminating connection at balancer level and resolving to one of the host from backend pool … all queriers always connect to frontend1 it shows status ready … other one shows no queriers connected

how to get around this problem ? do i need DNS that returns multiple A records ? i also checked push model of frontend instead pull (as it uses http) i.e. using
however it seems like with this approach frontned doesnt do queues , sharding nothing … it just proxies request to querier nodes … ? thats what doc says Query frontend example | Grafana Loki documentation so its like disabling frontend all together - i do not want that

Multiple DNS A records is probably your best bet.

1 Like

Alright thanks I was doing some research on load balancing over grpc and found it has some pitfalls because of persistence in grpc protocol
Will check with dns with multiple A records
I hope workers will be able connect to both frontends and
dns_lookup_duration parameter
in frontend_worker is honoured
As frontend will only be “ready” when some workers connect to it