Return-Path: 
 <dev-return-23732-apmail-kafka-dev-archive=kafka.apache.org@kafka.apache.org>
X-Original-To: apmail-kafka-dev-archive@www.apache.org
Delivered-To: apmail-kafka-dev-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 47DF110DEF
	for <apmail-kafka-dev-archive@www.apache.org>;
 Tue,  4 Nov 2014 17:07:44 +0000 (UTC)
Received: (qmail 36142 invoked by uid 500); 4 Nov 2014 17:07:43 -0000
Delivered-To: apmail-kafka-dev-archive@kafka.apache.org
Received: (qmail 36017 invoked by uid 500); 4 Nov 2014 17:07:43 -0000
Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:dev-help@kafka.apache.org>
List-Unsubscribe: <mailto:dev-unsubscribe@kafka.apache.org>
List-Post: <mailto:dev@kafka.apache.org>
List-Id: <dev.kafka.apache.org>
Reply-To: dev@kafka.apache.org
Delivered-To: mailing list dev@kafka.apache.org
Received: (qmail 35966 invoked by uid 99); 4 Nov 2014 17:07:42 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 04 Nov 2014 17:07:42 +0000
X-ASF-Spam-Status: No, hits=1.5 required=5.0
	tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of mistry.p.bhavesh@gmail.com
 designates 74.125.82.47 as permitted sender)
Received: from [74.125.82.47] (HELO mail-wg0-f47.google.com) (74.125.82.47)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 04 Nov 2014 17:07:16 +0000
Received: by mail-wg0-f47.google.com with SMTP id a1so14050255wgh.20
        for <multiple recipients>; Tue, 04 Nov 2014 09:05:44 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:date:message-id:subject:from:to:content-type;
        bh=Dwu8wFlEPeJHqHd8u+e4XN+fLQUBXhMtb58ux+3gDpw=;
        b=aUVVRcsfuxrPYPJHPAk/Gz3npIdFkFk9JNej0LZCgAWHxYTvUZlLV8lAyz9JeOrIpE
         oweG8eQEBS344pFsUiG/UJOxYZi6FX+c00Aq6ce7wLtnoz1kD4rGFoKMyOb/CLTUTevn
         ZgCKdAjOtr8vyohOuYn4AyIa1Ya1VPIPjaWAH27yxXTwuqsXChDdSUmEpCtsonbv8qA8
         hJqlelGZe6g3QJGMjuRXlajIGS7F9+ZKKmSb1g+ucuBWstewyqPKd1h5j1Ey1gN7EpJY
         cHRoUqrnrPGNjUAeW40/MR4dbyrzXGJun9rXlKGD/jaq3YElc+awNfR3iym+C03vyQPa
         72Ig==
MIME-Version: 1.0
X-Received: by 10.194.63.79 with SMTP id e15mr18414410wjs.79.1415120744680;
 Tue, 04 Nov 2014 09:05:44 -0800 (PST)
Received: by 10.217.48.134 with HTTP; Tue, 4 Nov 2014 09:05:44 -0800 (PST)
Date: Tue, 4 Nov 2014 09:05:44 -0800
Message-ID: 
 <CAG9AH6d-DcuvvBypO=17e2UJWFg2U9dRG4Cv9AbytzDG+VY2bA@mail.gmail.com>
Subject: queued.max.message.chunks impact and consumer tuning
From: Bhavesh Mistry <mistry.p.bhavesh@gmail.com>
To: "dev@kafka.apache.org" <dev@kafka.apache.org>,
 "users@kafka.apache.org" <users@kafka.apache.org>
Content-Type: multipart/alternative; boundary=047d7b86dada4d13ab05070b79d5
X-Virus-Checked: Checked by ClamAV on apache.org

--047d7b86dada4d13ab05070b79d5
Content-Type: text/plain; charset=UTF-8

Hi Kafka Dev Team,

It seems that Maximum buffer size is set to  2 default.  What is impact of
changing this to 2000 or so ?   This will improve the consumer thread
performance ?  More event will be buffered in memory.  Or Is there any
other recommendation to tune High Level Consumers ?

Here is code from Kafka Trunk Branch:

  val MaxQueuedChunks = 2
  /** max number of message chunks buffered for consumption, each chunk can
be up to fetch.message.max.bytes*/
  val queuedMaxMessages = props.getInt("queued.max.message.chunks",
MaxQueuedChunks)



Thanks,

Bhavesh

--047d7b86dada4d13ab05070b79d5--