I have quite simple queries which return data for 90 days without problem when used in explorer mode. But the same queries (some of them) ends with:
#### <html> <head><title>504 Gateway Time-out</title></head> <body> <center><h1>504 Gateway Time-out</h1></center> <hr><center>nginx</center> </body> </html>
when used on grafana dashboard.
Further inspection of failed query stats shows that it ends usually after 1 minute.
I have re-set various timeouts:
- grafana (loki datasource timeout 120s)
timeout = 300
- nginx (loki-distributed gateway)
but this happening still.
Did I forgot to something?