Return-Path: X-Original-To: apmail-flume-user-archive@www.apache.org Delivered-To: apmail-flume-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9404311160 for ; Sat, 20 Sep 2014 21:28:08 +0000 (UTC) Received: (qmail 32402 invoked by uid 500); 20 Sep 2014 21:28:08 -0000 Delivered-To: apmail-flume-user-archive@flume.apache.org Received: (qmail 32341 invoked by uid 500); 20 Sep 2014 21:28:08 -0000 Mailing-List: contact user-help@flume.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flume.apache.org Delivered-To: mailing list user@flume.apache.org Received: (qmail 32331 invoked by uid 99); 20 Sep 2014 21:28:08 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 20 Sep 2014 21:28:08 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of terreyshih@gmail.com designates 209.85.192.180 as permitted sender) Received: from [209.85.192.180] (HELO mail-pd0-f180.google.com) (209.85.192.180) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 20 Sep 2014 21:28:03 +0000 Received: by mail-pd0-f180.google.com with SMTP id r10so2004587pdi.25 for ; Sat, 20 Sep 2014 14:27:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:message-id:mime-version:subject:date:references :to:in-reply-to; bh=kdFxYqkOOIkkwCJRQBPwI3ERT6VJKop4sI5dqIrmBfI=; b=qu6LPyh+jcn+Y0cgRvS8wtLtCYlQdDxNpirABFa86MaCiS/yVi5fDP6tcBNSE9Ei2/ 9mj936k9zXZM5S583npItL7hfVJdDxTGyCmv5r5IghTBbRySQhItFVG3BJRrbB2OUFA/ omKWTbATzpDlaWylRuAIdqJqBY60bp1X1dRJIIAlON8Py6FpLbnMyBHfKrxg214/+HmT 83Iu/Kjj337RKWnAZXtQlPvArH3LE27O60w6ewSgDnceFcj2ntow6jOoomwoKLgE8MMu bpdqU08HX7hcnuM9ZK/BT8WqzaBNi+EYrSleRhO1gKS78j2x7HuRnV5tC8E7IKBWMuEl HwAw== X-Received: by 10.70.103.79 with SMTP id fu15mr14078546pdb.126.1411248462474; Sat, 20 Sep 2014 14:27:42 -0700 (PDT) Received: from [192.168.0.209] (99-162-148-175.lightspeed.sntcca.sbcglobal.net. [99.162.148.175]) by mx.google.com with ESMTPSA id tx8sm5367329pac.42.2014.09.20.14.27.41 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 20 Sep 2014 14:27:41 -0700 (PDT) From: terreyshih Content-Type: multipart/alternative; boundary="Apple-Mail=_F66A392A-E7D1-4E89-81BF-1F8AD95E0C42" Message-Id: <855EDE9A-7C43-4A26-A046-04CC21480A48@gmail.com> Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: dynamically change batch size in run time ? Date: Sat, 20 Sep 2014 14:27:38 -0700 References: To: user@flume.apache.org In-Reply-To: X-Mailer: Apple Mail (2.1878.6) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_F66A392A-E7D1-4E89-81BF-1F8AD95E0C42 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii thanks. On Sep 20, 2014, at 11:49 AM, Hari Shreedharan = wrote: > You can change batch size though - that is just config based. So you'd = have to reload the configuration. >=20 > On Sat, Sep 20, 2014 at 11:49 AM, Hari Shreedharan = wrote: > File Channel size cannot be changed dynamically - since the the File = Channel stores a fixed size queue on disk. Changing it means that you'd = have to delete the checkpoint and that would mean a full replay which = will take quite a while. >=20 > On Sat, Sep 20, 2014 at 8:16 AM, terreyshih = wrote: > Hi, >=20 > I know that flume reads/reloads the config file periodically, but does = it just check for changes in the data added (for example, new channel) = or does it also reload the batch size ? >=20 > I am doing some performance tuning and would like to know if I can = just change the batch size in run time to get some data >=20 > thx. >=20 >=20 --Apple-Mail=_F66A392A-E7D1-4E89-81BF-1F8AD95E0C42 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii thanks.

On Sep 20, 2014, at 11:49 = AM, Hari Shreedharan <hshreedharan@cloudera.com>= ; wrote:

You can change batch size though - that = is just config based. So you'd have to reload the = configuration.

On Sat, Sep 20, 2014 at 11:49 AM, Hari Shreedharan = <hshreedharan@cloudera.com> = wrote:
File = Channel size cannot be changed dynamically - since the the File Channel = stores a fixed size queue on disk. Changing it means that you'd have to = delete the checkpoint and that would mean a full replay which will take = quite a while.

On Sat, Sep 20, = 2014 at 8:16 AM, terreyshih <terreyshih@gmail.com> = wrote:
Hi,

I know that flume reads/reloads the config file periodically, but does = it just check for changes in the data added (for example, new channel) = or does it also reload the batch size ?

I am doing some performance tuning and would like to know if I can just = change the batch size in run time to get some data

thx.



= --Apple-Mail=_F66A392A-E7D1-4E89-81BF-1F8AD95E0C42--