How to configure nginx for frontend_worker?

Grafana Loki
1

I decided to introduce frontend into the current cluster and I would like to configure frontend_address in frontend_worker instead of downstream_url in frontend per the discussion in https://community.grafana.com/t/how-to-improve-loki-performance/42800.

Besides, I would like to put the 2 frontend services into nginx to make them load balancing, and here is the configuration of loki:

frontend:
  log_queries_longer_than: 5s
  #downstream_url: http://centos8:3100
  compress_responses: true

frontend_worker:
  frontend_address: centos8:9095
  parallelism: 12

centos8 is the server that nginx locates in and its config file(based on https://github.com/grafana/loki/blob/main/production/docker/config/nginx-loki-gateway.conf) is like:

upstream query-frontend {
      server centos8-3:9095;
      server centos8-4:9095;
    }

server {
    listen 9095 http2;
    location / {
        grpc_pass grpc://query-frontend;
    }

But once loki is spun up, error like below posted:

Aug 24 01:24:34 centos8-2 loki-linux-amd64[6725]: level=error ts=2021-08-23T17:24:34.719783648Z caller=frontend_processor.go:70 msg="error processing requests" address=192.168.209.133:9095 err="rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: PROTOCOL_ERROR"

I think there must be some mis-configuration in nginx, please guide me how to move on.

2

Though I added another upstream for port 3100 in nginx, it still failed.

3

Hi @dujas ,
Do you make it works? I also want to make the fronted service load balancing with nginx, but failed.

4

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.