Grafana Loki HTTP API: fails if I try to fetch more than 5,000 records

Dear team:
I am using Grafana Loki HTTP API to get information from Loki whilst I wait for the upcoming Loki Python client library :slightly_smiling_face:.

I noticed that my request to fetch records from Loki fails with HTTP 400 code when I request exactly any number greater than 5,000 records in the POST method.

The following is the payload I prepare to request the records from Loki:

┬┐ Is 5,000 a limit that can be enlarged?

I think this has nothing to do with Grafana┬┤s Loki data source menu, in which the following Maximum Lines parameter is already set to 1,000. If this parameter would have been related, then I would not have been able to correctly fetch up to 5,000 registers at once as I can for the time being.

Hints will be greatly appreciated!
Best regards


I think I found something. According to the documentationÔÇŽ

Batched queries

LogCLI sends queries to Loki such that query results arrive in batches.

The --limit option for a logcli query command caps the quantity of log lines for a single query. When not set, --limit defaults to 30. The limit protects the user from overwhelming the system for cases in which the specified query would have returned a large quantity of log lines. The limit also protects the user from unexpectedly large responses.

The quantity of log line results that arrive in each batch is set by the --batch option in a logcli query command. When not set, --batch defaults to 1000.

Setting a --limit value larger than the --batch value causes the requests from LogCLI to Loki to be batched. Loki has a server-side limit that defaults to 5000 for the maximum quantity of lines returned for a single query. The batching of requests allows you to query for a results set that is larger than the server-side limit, as long as the --batch value is less than the server limit.

Query metadata is output to stderr for each batch. Set the --quiet option on the logcli query command line to suppress the output of the query metadata.