flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Lord <jl...@cloudera.com>
Subject Re: "single source - multi channel" scenario and applying interceptor while writing to only one channel and not on others...possible approaches
Date Fri, 19 Apr 2013 18:14:04 GMT
Jagadish,

Here is an example of how to write a custom serializer.

https://github.com/apache/flume/blob/trunk/flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/MyCustomSerializer.java

-Jeff


On Fri, Apr 19, 2013 at 9:34 AM, Jeff Lord <jlord@cloudera.com> wrote:

> Hi Jagadish,
>
> Have you considered using a custom event serializer to modify your event?
> Its possible to replicate your flow using two channels and then have one
> sink that implements a custom serializer to modify the event.
>
> -Jeff
>
>
> On Tue, Apr 16, 2013 at 11:12 PM, Jagadish Bihani <
> jagadish.bihani@pubmatic.com> wrote:
>
>> Hi
>>
>> If anybody has any inputs on this that will surely help.
>>
>> Regards,
>> Jagadish
>>
>>
>> On 04/16/2013 12:06 PM, Jagadish Bihani wrote:
>>
>>> Hi
>>>
>>> We have a use case in which
>>> 1. spooling source reads data.
>>> 2. It needs to write events into multiple channels. It should apply
>>> interceptor only when putting into one channel and should put
>>> the event as it is while putting into another channel.
>>>
>>> Possible approach we have thought:
>>>
>>> 1. Create  2 different sources and then apply interceptor on one and dont
>>> apply on other. But that duplicates reads and increases IO.
>>>
>>> Is there any better way of achieving this use case?
>>>
>>> Regards,
>>> Jagadish
>>>
>>>
>>
>

Mime
View raw message