flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gonzalo Herreros <gherre...@gmail.com>
Subject Re: Flume interseptor and multiplexing
Date Tue, 16 Feb 2016 08:26:34 GMT
I don't think that's possible without writing/reusing custom code.
You would need an interceptor to add the header following the conditions
you describe so the multiplexer can do the routing.

Gonzalo

On 15 February 2016 at 21:06, chandra koripella <chandsks@gmail.com> wrote:

> Hi,
>
>
> Is there a way implement multiplexing when there is no header information
> attached to the event (based on first column)?
>
>
> we have customer data coming in in a file. There is no header information
> attached to the data. First column is even type(highlighted in red).
>
>
>
> ·        Each even type is bound to different metadata but it is not
> attached to incoming record.
>
> ·        There are 10 different event types.
>
> ·        Data needs to route to 10 different hdfs sinks.
>
> I am wondering if there is way to accomplish this without writing custom
> code.  I appreciate any help.
>
>
>
> 1|20160129010003|_abc|xxxxx@msn.com|31-EA1HYGT|31-EA1HYGF|R|1|xxx@msn.com
> <1%7C20160129010003%7C_BWqv-jB9KK8Nt4%7Cxxxxx@msn.com%7C31-EA1HYGT%7C31-EA1HYGF%7CR%7C1%7Cxxx@msn.com>
>
> 7|20160129010003|_aec|xxxx@aol.com|31-EA10ARI|31-EA10ARE|R|2|xxxx@aol.com
> <7%7C20160129010003%7C_BWqv-jB9KK8Nt4%7Cxxxx@aol.com%7C31-EA10ARI%7C31-EA10ARE%7CR%7C2%7Cxxxx@aol.com>
> |test|test|test
>
> 1
> 0|20160129010003|_asaasd|xxxx@aol.com|31-EA10ARI|31-EA10ARE|R|2|xxxx@aol.com|www.yahoo.com
> <10%7C20160129010003%7C_BWqv-jB9KK8Nt4%7Cxxxx@aol.com%7C31-EA10ARI%7C31-EA10ARE%7CR%7C2%7Cxxxx@aol.com%7Cwww.yahoo.com>
> |test
>
> 1
> |20160129010003|_rtert|xxxxx@msn.com|31-EA1HYGT|31-EA1HYGF|R|1|xxx@msn.com
> <1%7C20160129010003%7C_BWqv-jB9KK8Nt4%7Cxxxxx@msn.com%7C31-EA1HYGT%7C31-EA1HYGF%7CR%7C1%7Cxxx@msn.com>
>
> 4
> |20160129010003|_w3423|xxxxx@msn.com|31-EA1HYGT|31-EA1HYGF|R|1|xxx@msn.com|xx|xx|
> <4%7C20160129010003%7C_BWqv-jB9KK8Nt4%7Cxxxxx@msn.com%7C31-EA1HYGT%7C31-EA1HYGF%7CR%7C1%7Cxxx@msn.com%7Cxx%7Cxx%7C>
>
>
>
> Thanks,
>
> Chandra
>
>
>

Mime
View raw message