flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roshan Naik <ros...@hortonworks.com>
Subject Re: Problem Events
Date Wed, 24 Jul 2013 22:36:26 GMT
if you have a way to identify such events.. you may be able to use the
Regex interceptor to toss them out before they get into the channel.

On Wed, Jul 24, 2013 at 2:52 PM, Jeremy Karlson <jeremykarlson@gmail.com>wrote:

> Hi everyone.  My Flume adventures continue.
> I'm in a situation now where I have a channel that's filling because a
> stubborn message is stuck.  The sink won't accept it (for whatever reason;
> I can go into detail but that's not my point here).  This just blocks up
> the channel entirely, because it goes back into the channel when the sink
> refuses.  Obviously, this isn't ideal.
> I'm wondering what mechanisms, if any, Flume has to deal with these
> situations.  Things that come to mind might be:
> 1. Ditch the event after n attempts.
> 2. After n attempts, send the event to a "problem area" (maybe a different
> source / sink / channel?)  that someone can look at later.
> 3. Some sort of mechanism that allows operators to manually kill these
> messages.
> I'm open to suggestions on alternatives as well.
> Thanks.
> -- Jeremy

View raw message