Return-Path: 
 <users-return-3229-apmail-kafka-users-archive=kafka.apache.org@kafka.apache.org>
X-Original-To: apmail-kafka-users-archive@www.apache.org
Delivered-To: apmail-kafka-users-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 62E31D336
	for <apmail-kafka-users-archive@www.apache.org>;
 Wed, 23 Jan 2013 03:15:19 +0000 (UTC)
Received: (qmail 29797 invoked by uid 500); 23 Jan 2013 03:15:19 -0000
Delivered-To: apmail-kafka-users-archive@kafka.apache.org
Received: (qmail 29722 invoked by uid 500); 23 Jan 2013 03:15:18 -0000
Mailing-List: contact users-help@kafka.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:users-help@kafka.apache.org>
List-Unsubscribe: <mailto:users-unsubscribe@kafka.apache.org>
List-Post: <mailto:users@kafka.apache.org>
List-Id: <users.kafka.apache.org>
Reply-To: users@kafka.apache.org
Delivered-To: mailing list users@kafka.apache.org
Received: (qmail 29708 invoked by uid 99); 23 Jan 2013 03:15:18 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 23 Jan 2013 03:15:18 +0000
X-ASF-Spam-Status: No, hits=1.7 required=5.0
	tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of sahmed1020@gmail.com designates
 209.85.216.45 as permitted sender)
Received: from [209.85.216.45] (HELO mail-qa0-f45.google.com) (209.85.216.45)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 23 Jan 2013 03:15:10 +0000
Received: by mail-qa0-f45.google.com with SMTP id bv4so268079qab.11
        for <users@kafka.apache.org>; Tue, 22 Jan 2013 19:14:50 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=x-received:mime-version:from:date:message-id:subject:to
         :content-type;
        bh=xguYDqJCim4ufHgEIfiNNrrhpSIynS7pjvPwBADLi6E=;
        b=PigK0NkJ8LD2IJMjOtpt8x+i/q+69k72VdRuWEz3mt9hwYi9XxJxCNtW/9E96A+lLl
         ooJpvhGF9jLjPaJQEFc0ngZ4k1fNjFJUd67Tn7e0VyZhxNtFv1fiVCpCljLURtZnR7Pj
         1HOqzVADflMjYUAOETr1AxL0wSq2JoYtIVy0NukEdHxa/WeB/7pEurFZ1g6M3TuQQ/o4
         jZAP/PmN1Aj+h6i64W8MjTQMEwDUGGr+Ru33wU4epWj6DeE1USvAhWgBCWeg/IRoTgEY
         UBS5Xg2PiHhYGLkiGZ7zWHezmhDzQx1uJaIlSpR32jH/yULVplzrujkf1AEnsboSkrOY
         Jrew==
X-Received: by 10.49.71.204 with SMTP id x12mr29065116qeu.47.1358910890072;
 Tue, 22 Jan 2013 19:14:50 -0800 (PST)
MIME-Version: 1.0
Received: by 10.49.26.37 with HTTP; Tue, 22 Jan 2013 19:14:30 -0800 (PST)
From: S Ahmed <sahmed1020@gmail.com>
Date: Tue, 22 Jan 2013 22:14:30 -0500
Message-ID: 
 <CAG2rwuM15PrdwisvUTtPSDxH3hL5_ZKEmi==iuKcdZQbM-QGvg@mail.gmail.com>
Subject: java, oom, gc stop the world
To: users@kafka.apache.org
Content-Type: multipart/alternative; boundary=047d7b6d8672e2594f04d3ec1820
X-Virus-Checked: Checked by ClamAV on apache.org

--047d7b6d8672e2594f04d3ec1820
Content-Type: text/plain; charset=ISO-8859-1

In the wild, what sort of memory usage patterns have you guys seen with
kafka?

I'm not that well versed with java and its memory/gc issues, what have you
guys experienced with your production brokers?  (assuming consumers and
producers are not that much of an issue).

How do you determine how much memory to allocate to a given broker?  Is it
based on factors such as:

1. producers are mostly using batch versus single messages
2. messages per second
3. size of message payload
4. # of consumers connecting to a given broker
etc.

Is there a "magic" forumula?

--047d7b6d8672e2594f04d3ec1820--