Only One Loki Instance Can Be Deploy In One Machine In Cluster Mode

Grafana Loki
1

Hi All,
I try to setup an cluster with 2 node(instance) in a machine. The configuration is really simple:
Instance 1

auth_enabled: true

server:
  http_listen_port: 3100
  grpc_listen_port: 9096

memberlist:
  join_members:
    - localhost:7946
  bind_port: 7946

common:
  path_prefix: /tmp/loki_1
  storage:
    filesystem:
      chunks_directory: /tmp/loki_1/chunks
      rules_directory: /tmp/loki_1/rules
  replication_factor: 1
  ring:
    kvstore:
      store: memberlist

schema_config:
...

ruler:
  alertmanager_url: http://localhost:9093

Instance 2

auth_enabled: true

server:
  http_listen_port: 13100
  grpc_listen_port: 19096

memberlist:
  join_members:
    - localhost:7946
  bind_port: 17946

common:
  path_prefix: /tmp/loki_2
  storage:
    filesystem:
      chunks_directory: /tmp/loki_2/chunks
      rules_directory: /tmp/loki_2/rules
  replication_factor: 1
  ring:
    kvstore:
      store: memberlist

schema_config:
...

ruler:
  alertmanager_url: http://localhost:19093

After run loki base on above config, I expect 2 active, but the metric data only show 1 active:

cortex_ring_members{name="compactor",state="ACTIVE"} 1
cortex_ring_members{name="compactor",state="JOINING"} 0
cortex_ring_members{name="compactor",state="LEAVING"} 0
cortex_ring_members{name="compactor",state="PENDING"} 0
cortex_ring_members{name="compactor",state="Unhealthy"} 0
cortex_ring_members{name="ingester",state="ACTIVE"} 1
cortex_ring_members{name="ingester",state="JOINING"} 0
cortex_ring_members{name="ingester",state="LEAVING"} 0
cortex_ring_members{name="ingester",state="PENDING"} 0
cortex_ring_members{name="ingester",state="Unhealthy"} 0
cortex_ring_members{name="scheduler",state="ACTIVE"} 1
cortex_ring_members{name="scheduler",state="JOINING"} 0
cortex_ring_members{name="scheduler",state="LEAVING"} 0
cortex_ring_members{name="scheduler",state="PENDING"} 0
cortex_ring_members{name="scheduler",state="Unhealthy"} 0

But if I deploy instance 2 to difference machine, it will show 2 active:

cortex_ring_members{name="compactor",state="ACTIVE"} 2
cortex_ring_members{name="compactor",state="JOINING"} 0
cortex_ring_members{name="compactor",state="LEAVING"} 0
cortex_ring_members{name="compactor",state="PENDING"} 0
cortex_ring_members{name="compactor",state="Unhealthy"} 0
cortex_ring_members{name="ingester",state="ACTIVE"} 2
cortex_ring_members{name="ingester",state="JOINING"} 0
cortex_ring_members{name="ingester",state="LEAVING"} 0
cortex_ring_members{name="ingester",state="PENDING"} 0
cortex_ring_members{name="ingester",state="Unhealthy"} 0
cortex_ring_members{name="scheduler",state="ACTIVE"} 2
cortex_ring_members{name="scheduler",state="JOINING"} 0
cortex_ring_members{name="scheduler",state="LEAVING"} 0
cortex_ring_members{name="scheduler",state="PENDING"} 0
cortex_ring_members{name="scheduler",state="Unhealthy"} 0

Is it for cluster case, only one loki instance can be deploy in one machine, or I make anything wrong?

2

After some investigation, I found the reason, most of the component has id, the id should be unique for each instance of component, when you run multiple instance of component in the same machine, then you need to explicitly set id, otherwise only one instance component is add to the cluster.

3

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.