Code(429) desc = Maximum active stream limit exceeded, reduce the number of active streams (reduce labels or reduce label values), or contact your Loki administrator to see if the limit can be increased" msg=gRPC

Grafana Configuration
,
1

Grafana version ：2.5.0

promtail is going to send some stream to loki but get the error message.

Confugration
here s my promtail config by matching ‘special’ job:

- match:
            selector: '{job="special"}'
            stages:
            - multiline:
                firstline: '^\[[ZT\d:\+-\.]*\]'
                max_wait_time: 3s
                max_lines: 100000

            - regex:
                expression: '^\[(?P<time2>[ZT\d:\+-\.]*)\]--\[(?P<serviceName>[-_\w\d]*)\]--\[(?P<logType>[-_\w\d]*)\]--.*\"serialNumber\":\"(?P<serialNumber>[-_\w\d]*)\".*\"deviceName\":\"(?P<deviceName>[^\\"]*)\".*\"eventType\":\"(?P<eventType>\[.*\])\".*'
            - labels:
                serviceName:
                logType:
                serialNumber:
                deviceName:
                eventType:

Appearance：
Before，I collect this job by json label，I change my config later as I mentioned (to support some big log) and I save some static labels those I care(serialNumber,deviceName,eventType).

However, those labels make the error message —— Code(429) desc = Maximum active stream limit exceeded, reduce the number of active streams (reduce labels or reduce label values), or contact your Loki administrator to see if the limit can be increased" msg=gRPC

Handle
Then I post the loki api “/loki/api/v1/label/serialNumber/values” to see how much values did I collect in the ‘serialNumber’ label, and it’s only 120 values.

Question
I want to know which configuration control the number of active streams in promtail or loki?

thanks for looking this question.
I would be very grateful for ur answer

2

Did you ever found a solution for your 429 issue.
We are facing something simalar, but we are not using promtail, but fluentd and fluent-bit.

3

I set this config in ingester:

limits_config:

limits_config:
**max_global_streams_per_user: 0

u can find the description about the configs in Loki git project:

Maximum number of active streams per user, across the cluster. 0 to disable.
When the global limit is enabled, each ingester is configured with a dynamic
local limit based on the replication factor and the current number of healthy
ingesters, and is kept updated whenever the number of ingesters change.
CLI flag: -ingester.max-global-streams-per-user
[max_global_streams_per_user: | default = 5000]

I think that could fix the error 429, and I did it.
But I found that too many streams ( maybe labels ) would be easy to increase memory or cpu resource.

u can try

1 Like