flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohit Anchlia <mohitanch...@gmail.com>
Subject Re: Memory Channel
Date Wed, 16 Jan 2013 18:47:52 GMT
Right. I was asking about sync to "sink". My sink is hdfs so does flume
sync to hdfs on every write operation?

On Wed, Jan 16, 2013 at 10:26 AM, Brock Noland <brock@cloudera.com> wrote:

> Memory Channel does not write to disk and as such never syncs to disk.
> File Channel does sync to disk for each batch put on or taken off the
> channel.
>
> On Wed, Jan 16, 2013 at 10:21 AM, Mohit Anchlia <mohitanchlia@gmail.com>
> wrote:
> > Thanks! What I am really trying to understand is when does flume sync to
> the
> > sink. I am not using batch events.
> >
> >
> > On Wed, Jan 16, 2013 at 9:55 AM, Hari Shreedharan
> > <hshreedharan@cloudera.com> wrote:
> >>
> >> It means that the channel can store that many events. If it is full,
> then
> >> the put() calls (on the source side) will start throwing
> ChannelException.
> >> The put call will block only for keep-alive number of seconds, after
> which
> >> it will throw.
> >>
> >>
> >> Hari
> >>
> >> --
> >> Hari Shreedharan
> >>
> >> On Wednesday, January 16, 2013 at 9:46 AM, Mohit Anchlia wrote:
> >>
> >> Could someone help me understand capacity attribute of memoryChannel?
> Does
> >> it mean that memoryChannel flushes to sink only when this capacity is
> >> reached or does it mean that it's the max events stored in memory and
> call
> >> blocks until everything else gets freed?
> >>
> >>
> >> http://flume.apache.org/FlumeUserGuide.html#memory-channel
> >>
> >>
> >>
> >
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce -
> http://incubator.apache.org/mrunit/
>

Mime
View raw message