504 Gateway Time-out nginx/1.27.2 Timeout Issues When Using Large Query Limits

Hello,

I’m encountering the following error in Grafana when using Loki as the data source:

504 Gateway Time-out

504 Gateway Time-out


nginx/1.27.2

The full error message looks like this:

<html>
  <head><title>504 Gateway Time-out</title></head>
  <body>
    <center><h1>504 Gateway Time-out</h1></center>
    <hr><center>nginx/1.27.2</center>
  </body>
</html>

This error occurs when I try to execute large queries on my Loki data source. Specifically, I see this issue when I attempt to query for a larger set of logs (e.g., 10,000 entries), while smaller queries (like 1,000 entries) execute successfully.

  1. Has anyone encountered the 504 Gateway Time-out error with Loki as a data source and found a resolution?
  2. Reduced the query limit to 1,000, which works fine, but I need a solution to handle larger queries (e.g., 10,000 logs).

Processing of large set needs more time. It looks like you are hitting some timeout limits. Make sure that your infrastructure (e.g. your Nginx/Grafana/Loki/…) has configured timeouts, which are sufficient for your query.

There is many timeouts, so check the doc and configure those timeouts, which are applicable for you setup. For example random post about nginx timeouts.
https://www.baeldung.com/linux/nginx-timeouts