kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "radai rosenblatt (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-4011) allow sizing RequestQueue in bytes
Date Mon, 01 Aug 2016 21:56:20 GMT
radai rosenblatt created KAFKA-4011:
---------------------------------------

             Summary: allow sizing RequestQueue in bytes
                 Key: KAFKA-4011
                 URL: https://issues.apache.org/jira/browse/KAFKA-4011
             Project: Kafka
          Issue Type: Improvement
          Components: core
    Affects Versions: 0.10.0.0
            Reporter: radai rosenblatt
             Fix For: 0.10.1.0


currently RequestChannel's requestQueue is sized in number of requests:

{code:title=RequestChannel.scala|borderStyle=solid}
private val requestQueue = new ArrayBlockingQueue[RequestChannel.Request](queueSize)
{code}

under the assumption that the end goal is a bound on server memory consumption, this requires
the admin to know the avg request size.

I would like to propose sizing the requestQueue not by number of requests, but by their accumulated
size (Request.buffer.capacity). this would probably make configuring and sizing an instance
easier.

there would need to be a new configuration setting for this (queued.max.bytes?) - which could
be either in addition to or instead of the current queued.max.requests setting



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message