Has anyone got an example configuration for running an ingester, distributor, and querier as microservices on the same server using memberlist as the ring store?
Using consul I have this working - well, logs get to S3, and I can query them. However, it would be good to have a zero-dependencies setup, so we’d like to use memberlist for ring storage. With consul, I can browse to /ring, and see the cortex ring status, with 3 members (I’m running the a set of ingester/distributor/querier on each of 3 VMs).
If I try to use memberlist, I constantly get errors about the ring being empty, and the /ring url shows no members.
In the ingester.yaml, I have this:
which is the same on all 3 servers, and for the distributor.yaml, I have the same stanza. All 3 components run on each of 3 servers.
Only the ingester process shows port 7946 - the (bind port for the ring) in the output of ss -ntlp, irrespective of changing the bind address in the other component config files, which is actually what I expected, but I also expected that the other components would join the ring by talking to that port. I’m very confused, and suspect that a working example using memnerlist - even for a single component to VM, but with more than one vm.
By running 2 ingesters on one host (with different ports), I have come to the conclusion that the memberlist ring does not update the /ring URL - would that be correct.