flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom Wheeler <tomwh...@gmail.com>
Subject Confused About Use of Transactions in Custom Source
Date Sun, 12 Jan 2014 15:42:49 GMT
I'm confused about the proper way to use transactions in a custom source,
as the documentation and examples seem to disagree with one another. I'm
hoping someone could let me know the recommended approach.

The custom source in the Flume Developer's guide shows the use of a
transaction in the process() method; specifically, it stores the event in
the channel via the ChannelProcessor's processEvent method, just before it
commits the transaction:

     http://flume.apache.org/FlumeDeveloperGuide.html#source

It looks to me. however, like ChannelProcessor.processEvent method itself
wraps everything in a transaction:


http://svn.apache.org/repos/asf/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/channel/ChannelProcessor.java

Given that, does the custom source *really* need its own transaction like
the example shows? I notice other sources that do not use one:


http://svn.apache.org/repos/asf/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/source/SyslogTcpSource.java

http://svn.apache.org/repos/asf/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java

http://svn.apache.org/repos/asf/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java

Thanks,

Tom

Mime
View raw message