flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom Wheeler <tomwh...@gmail.com>
Subject Re: Confused About Use of Transactions in Custom Source
Date Mon, 13 Jan 2014 00:35:42 GMT
Just to close the loop on this, a Flume developer confirmed separately that
the transaction in the source is unnecessary, so I filed FLUME-2297
regarding the unnecessary transaction in the developer guide.

On Sun, Jan 12, 2014 at 9:42 AM, Tom Wheeler <tomwheel@gmail.com> wrote:

> I'm confused about the proper way to use transactions in a custom source,
> as the documentation and examples seem to disagree with one another. I'm
> hoping someone could let me know the recommended approach.
> The custom source in the Flume Developer's guide shows the use of a
> transaction in the process() method; specifically, it stores the event in
> the channel via the ChannelProcessor's processEvent method, just before it
> commits the transaction:
>      http://flume.apache.org/FlumeDeveloperGuide.html#source
> It looks to me. however, like ChannelProcessor.processEvent method itself
> wraps everything in a transaction:
> http://svn.apache.org/repos/asf/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/channel/ChannelProcessor.java
> Given that, does the custom source *really* need its own transaction like
> the example shows? I notice other sources that do not use one:
> http://svn.apache.org/repos/asf/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/source/SyslogTcpSource.java
> http://svn.apache.org/repos/asf/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java
> http://svn.apache.org/repos/asf/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java
> Thanks,
> Tom

View raw message