Return-Path: X-Original-To: apmail-flume-user-archive@www.apache.org Delivered-To: apmail-flume-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 18285EC06 for ; Wed, 16 Jan 2013 18:48:21 +0000 (UTC) Received: (qmail 67472 invoked by uid 500); 16 Jan 2013 18:48:20 -0000 Delivered-To: apmail-flume-user-archive@flume.apache.org Received: (qmail 67427 invoked by uid 500); 16 Jan 2013 18:48:20 -0000 Mailing-List: contact user-help@flume.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flume.apache.org Delivered-To: mailing list user@flume.apache.org Received: (qmail 67418 invoked by uid 99); 16 Jan 2013 18:48:20 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 16 Jan 2013 18:48:20 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of mohitanchlia@gmail.com designates 209.85.220.172 as permitted sender) Received: from [209.85.220.172] (HELO mail-vc0-f172.google.com) (209.85.220.172) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 16 Jan 2013 18:48:14 +0000 Received: by mail-vc0-f172.google.com with SMTP id l6so617166vcl.31 for ; Wed, 16 Jan 2013 10:47:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type; bh=qTCsLSxE+tX4Pn9vxBWgjrdVDJX3URrLNxbgAP/GAgU=; b=LEZDLL0+EVKmr88QAA/STW99gcLO0eRZ3552EIt1eM0Jq/WB1Pc6x28M/cgsIR2aUG jxBRcgAMcsoEHFmEoYy4QWKEtxFKMuOA8uK8DiigiEEubfpy9aMtU2QfGejUh30t01gN DU50Wr9rdMd1OUqVuKFO+IF/QMXJhQSi1AgmuDiRiie0KLLA3oiRE982s0zcl8Azmea2 oFcw+W/ULxiZmjRBn5AX3u6tkPIgbAsqqLfe4unew53jN4D/oykLazzOopupgr8PAYIk 7ZYhMsitHbh4okwLSFS4pVegZsTOdVB2ZNeZhsP+1pas77JbVwQqJuMyqNkEFwxayJ1w CsYA== MIME-Version: 1.0 X-Received: by 10.52.16.229 with SMTP id j5mr2077056vdd.111.1358362073088; Wed, 16 Jan 2013 10:47:53 -0800 (PST) Received: by 10.220.144.5 with HTTP; Wed, 16 Jan 2013 10:47:52 -0800 (PST) In-Reply-To: References: <29A350075FD747C08EF0B7DCFE4BB867@cloudera.com> Date: Wed, 16 Jan 2013 10:47:52 -0800 Message-ID: Subject: Re: Memory Channel From: Mohit Anchlia To: user@flume.apache.org Content-Type: multipart/alternative; boundary=bcaec502dbbed7c3a904d36c5032 X-Virus-Checked: Checked by ClamAV on apache.org --bcaec502dbbed7c3a904d36c5032 Content-Type: text/plain; charset=ISO-8859-1 Right. I was asking about sync to "sink". My sink is hdfs so does flume sync to hdfs on every write operation? On Wed, Jan 16, 2013 at 10:26 AM, Brock Noland wrote: > Memory Channel does not write to disk and as such never syncs to disk. > File Channel does sync to disk for each batch put on or taken off the > channel. > > On Wed, Jan 16, 2013 at 10:21 AM, Mohit Anchlia > wrote: > > Thanks! What I am really trying to understand is when does flume sync to > the > > sink. I am not using batch events. > > > > > > On Wed, Jan 16, 2013 at 9:55 AM, Hari Shreedharan > > wrote: > >> > >> It means that the channel can store that many events. If it is full, > then > >> the put() calls (on the source side) will start throwing > ChannelException. > >> The put call will block only for keep-alive number of seconds, after > which > >> it will throw. > >> > >> > >> Hari > >> > >> -- > >> Hari Shreedharan > >> > >> On Wednesday, January 16, 2013 at 9:46 AM, Mohit Anchlia wrote: > >> > >> Could someone help me understand capacity attribute of memoryChannel? > Does > >> it mean that memoryChannel flushes to sink only when this capacity is > >> reached or does it mean that it's the max events stored in memory and > call > >> blocks until everything else gets freed? > >> > >> > >> http://flume.apache.org/FlumeUserGuide.html#memory-channel > >> > >> > >> > > > > > > -- > Apache MRUnit - Unit testing MapReduce - > http://incubator.apache.org/mrunit/ > --bcaec502dbbed7c3a904d36c5032 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Right. I was asking about sync to "sink". My sink is hdfs so does= flume sync to hdfs on every write operation?

On Wed, Jan 16, 2013 at 10:26 AM, Brock Noland <= span dir=3D"ltr"><brock@cloudera.com> wrote:
Memory Channel does not write to disk= and as such never syncs to disk.
File Channel does sync to disk for eac= h batch put on or taken off the
channel.

On Wed, Jan 16, 2013 at 10:21 AM, Mohit Anchlia <<= a href=3D"mailto:mohitanchlia@gmail.com">mohitanchlia@gmail.com> wro= te:
> Thanks! What I am really trying to understand is when does flum= e sync to the
> sink. I am not using batch events.
>
>
> On Wed, Jan= 16, 2013 at 9:55 AM, Hari Shreedharan
> <hshreedharan@cloudera.com> wrote:
>> >> It means that the channel can store that many events. If it is ful= l, then
>> the put() calls (on the source side) will start throwin= g ChannelException.
>> The put call will block only for keep-alive= number of seconds, after which
>> it will throw.
>>
>>
>> Hari
>>= ;
>> --
>> Hari Shreedharan
>>
>> On We= dnesday, January 16, 2013 at 9:46 AM, Mohit Anchlia wrote:
>>
>> Could someone help me understand capacity attribute of memoryChann= el? Does
>> it mean that memoryChannel flushes to sink only when t= his capacity is
>> reached or does it mean that it's the max e= vents stored in memory and call
>> blocks until everything else gets freed?
>>
>>>> http://flume.apache.org/FlumeUserGuide.html#memory= -channel
>>
>>
>>
>



--
Apache MRUnit - Unit testing = MapReduce - http://incubator.apache.org/mrunit/

--bcaec502dbbed7c3a904d36c5032--