flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guillermo Ortiz <konstt2...@gmail.com>
Subject How to connect two flows with specific behavior.
Date Mon, 28 Jul 2014 11:13:25 GMT
I want to create a topology for Flume, what I want to get it's,.

Data---> Source1-->Channel1-->MySink1 --->Source2 --> Channel2/Channel3

Channel2 --> SinkHDFS
Channel3 --> MySinkHBase

I'd need to code MySink1 and do an special transformation to my data, the
output would be the input for Source2.
Finally, these data should store in Hdfs with the standard sink of flume
and HBase, where I should create a new Serializer for HBase or something
like that.

I can't see how to do the connection between MySink1 and Source2.
Should Source2 be of Avro type? I think that if I want to connect many
flows inside Flume they have to be Avro, How I want an specific behavior, I
should create a new implementation which extends AbstractRpcAvro or
something like that... Am I right?

View raw message