flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Schneider <ch...@christopher-schneider.com>
Subject Ordering of messages in flume-ng
Date Wed, 12 Feb 2014 05:52:52 GMT
I've seen a fair number of resources on the web that describe the loose
ordering guarantees that flume offers for messages in the face of
degradation or failures.  But I can't tell what applies to flume-og, and
flume-ng.  Hopefully somebody can help clear up the situation.

In the case of a single agent topology, (source -> FileSystem Channel ->
sink), can messages become out of order?  What situations cause that?

In a multi agent topology, does that answer change?

(Agent 1   Source -> FilesystemChannel -> Avro To Collector)
(Agent 2   Source -> FilesystemChannel -> Avro To Collector)
(Collector Avro from agents -> FilesystemChannel -> final Sink)

And perhaps in an even more complicated setup, with multiple collectors,
does that answer change further?

View raw message