504 gateway timeout (all configs look good)

[gwestrelin]

Hello,

My grafana is on a K8S cluster. It was installed by Official helm Chart, with Prometheus, Loki, Promtail…
In Loki, timeout for querier is 10 minutes.
In Grafana, dataproxy.timeout is 60 second.
In Traefik ingress controller, timeout is 3 minutes (by default)

So everything seems good in the configurations but when I do a search for more than 10 days, I occasionally get a “504 Gateway Time-out” error quite quickly (4 or 5 seconds) that I don’t have below of 10 days.

Can you help me please ?
I don’t know where the problem can come from

Thanks you a lot !
Regards,

[usman.ahmad]

Hi @gwestrelin,

Welcome to the community support forums !!

Well, I am little bit inexperienced on the K8s world but I believe there are many experienced users in the community that might be able to help you out.

The best way will be to do the following:

1- Share your configuration files (If possible)
2- paste the error from the logs so that can help to understand the issue better (can increase the verbosity if it does not show at the moment).