//抛出429 Too Many Requests异常 return nil, httpgrpc.Errorf(http.StatusTooManyRequests, validation.RateLimitedErrorMsg(int(d.ingestionRateLimiter.Limit(now, userID)), validatedSamplesCount, validatedSamplesSize)) } 接下来就是具体的限流器内的处理,我们看到Distributor主要通过RateLimiter.AllowN的方式获...
level=warn ts=2022-02-18T09:41:39.186511145Z caller=client.go:349component=client host=loki:3100msg="error sending batch, will retry"status=429error="server returned HTTP status 429 Too Many Requests (429): Maximum active stream limit exceeded, reduce the number of active streams (reduce labe...
userID).Add(float64(validatedSamplesCount))validation.DiscardedBytes.WithLabelValues(validation.RateLimited,userID).Add(float64(validatedSamplesSize))//抛出429 Too Many Requests异常returnnil,httpgrpc.Errorf(http.StatusTooManyRequests,validation.RateLimitedErrorMsg(int(...
userID).Add(float64(validatedSamplesCount))validation.DiscardedBytes.WithLabelValues(validation.RateLimited,userID).Add(float64(validatedSamplesSize))//抛出429 Too Many Requests异常returnnil,httpgrpc.Errorf(http.StatusTooManyRequests,validation.RateLimitedErrorMsg(int(...
429 Too Many Requests Ingestion rate limit exceeded (limit 10485760 bytes/sec) 1. 这直接导致我们部分服务器上的日志采集器缓冲区一直处于拥堵,引起日志采集的延迟。 看起来这个是触发了Loki的限流策略了,查了下官方文档,发现在limits_config中两个参数控制Loki Distributor的日志接收的速率: ...
However, almost immediately after the start of promtail, I see the following messages: msg=“error sending a packet, I will try again” status=429 error=“the server returned the HTTP status 429 Too many requests (429): The maximum limit of the active stream has been exceeded, reduce the...
HTTP status code429 Too Many Requests stream_limit This limit is enforced when a tenant reaches their maximum number of active streams. Active streams are held in memory buffers in the ingesters, and if this value becomes sufficiently large then it will cause the ingesters to run out of me...
随后我们依次启动promtail和loki即可,注意,第一次启动,会扫描匹配的所有完整日志,因为没有记录的日志偏移量,这样的造成的影响是,会发现出现大量429报错,server returned HTTP status 429 Too Many Requests (429),我的处理方法是第一次采集完成后,等待promtail扫描到不报错为止,然后停止loki服务,将loki的数据清空,重...
上述配置限制了每个租户的速率为每秒100个请求,且允许短时期内的最大突发请求数为10个。如果超过了这个限制,Loki将返回429“Too Many Requests”错误。 Loki可以限制每秒钟的数据大小吗? 在Loki中,限制日志事件速率的方式通常使用每秒事件数来进行配置,而不是使用数据的大小来进行配置。这是因为日志事件的大小可能会因...
429 Too Many Requests Ingestion rate limit exceeded (limit: 8388608 bytes/sec) while attempting to ingest '2140' lines totaling '3285284' bytes 429 Too Many Requests Ingestion rate limit exceeded' or '500 Internal Server Error rpc error: code = ResourceEx...