We currently managing a loki cluster and monitoring it using prometheus.
We are thinking about monitoring the loki components with promtail as well, so we can follow the logs, if one of the component fall we could look into it deeper using it, but is it right to do so?
It will not create an infinite loop of logs because neither the distributor nor the ingester are logging after getting new data from the clients.
It will work, yes.
As long as you’re not shipping Promtail’s logs to Loki as well, you’ll avoid the infinite loop
Also, be sure to add some limits as an extra protection against overloading Loki with its own logs