flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hari Shreedharan <hshreedha...@cloudera.com>
Subject Re: Estimating the event loss
Date Fri, 17 Jul 2015 18:20:29 GMT
Note that if you simply reconfigure (rather than kill the JVM and restart)
you would not lose data in the memory channel


Thanks,
Hari

On Fri, Jul 17, 2015 at 7:08 AM, Johny Rufus <jrufus@cloudera.com> wrote:

> When you restart the agent, all the events in the memory channel will be
> lost (if the memory channel was full and contained 100k events, they will
> be lost)
> File channel will not result in loss of events, on restart the channel's
> state will be restored (File channel always maintains the last two data
> files the events were written to, even if the events in them were
> completely drained, this may give the impression that it still has data
> left)
>
> Thanks,
> Rufus
>
> On Thu, Jul 16, 2015 at 11:10 PM, Buntu Dev <buntudev@gmail.com> wrote:
>
>> I'm using a Memory channel with capacity set to 100000. Does this mean
>> when the flume agent restarts, its possible that I loose about 100k events?
>>
>> For any other durable channel say File channel I noticed .tmp files
>> created and written to but when I restart the agent these .tmp files are
>> left as-is. Since some events seem to be written to the .tmp files, should
>> I expect loss of events in this scenario?
>>
>>
>> Thanks!
>>
>
>

Mime
View raw message