could someone direct me to a readme on proper server architecture for loki instances? I have multiple windows nodes, that I need to be able to query from a single url. Not sure whether to run a loki instance on each node, then forward …somewhere. Or do something else. the nodes run dotnetcore apps, not containers.
you don’t need to install loki on every node.
In general you need it loki installed on one node (server) and then you need to install promtail on all of them. Promtail send logs to loki.
You have a bunch of tutorials and good text how you can do that on net. Try to search sentence like:
" How to Setup Promtail, Grafana and Loki"
I hope I help you a little bit.
thanks @petarkozic. I’m looking for a use-case where there is no central server. We have a collection of nodes with no master (only a load balancer).
can I use a single azure storage blob, send all logs to that storage, and then query the singular storage from any loki/grafana instance?
You can’t use same folder on storage for all Loki instances but you can use same storage with different folder names, becasue in Loki configuration you need to define folder name where Loki stores chunks.
On example on one Azure storage you will have folders like:
loki1, loki2, loki3
You can connect grafana with all loki servers, that mean if you have five grafana server you can connect all loki server with all of them. And answer on your question is here
In general, I don’t know why you need to have more grafana servers. One is enough because you can connect all loki servers in one grafana and make query on all of them.
On example, I have two Prometheus servers and one Loki server and all of them are connected with one Grafana server and I can query and make dashboards from one Grafana.
One grafana instance which can read multiple loki endpoints would be great. If grafana could leverage a service discovery endpoint to be able to find all the loki endpoints, then it would be perfect.