incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ariel Rabkin <asrab...@gmail.com>
Subject Re: Custom Processor
Date Fri, 12 Feb 2010 00:49:06 GMT
Change sounds fine to me...let's do it.

On Thu, Feb 11, 2010 at 3:52 PM, Eric Yang <eyang@yahoo-inc.com> wrote:
> I am ok with this.  Ari, comment?
>
> Regards,
> Eric
>
> On 2/11/10 3:49 PM, "Corbin Hoenes" <corbin@tynt.com> wrote:
>
>> This maybe a bit of a nit pick but what about instead of calling this
>> nextLine() it was called nextRecord()?
>>
>> On Feb 5, 2010, at 12:57 PM, Eric Yang wrote:
>>
>>> Log output by ChukwaLog4J appender has a special property that it will group
>>> the same log output statement delimited by ^A character.  Hence, the UTF8
>>> adaptor will not break up the structures and send the entire output as one
>>> record.  When the custom processor get an recordEntry (part of a chunk), it
>>> will contain the full output.  nextLine function is to get the next record
>>> shipped in the same chunk.  It should not be rely upon, unless the specially
>>> crafted adaptor is shipping multi-records as a chunk and this chunk has no
>>> dependency outside.  The current UTF8 adaptor is sending multiple records
>>> depending on the speed of the tail is happening.  UTF8 adaptor should not be
>>> rely upon if you want to use nextLine feature to build your parser states.
>>>
>>> Regards,
>>> Eric
>>>
>>> On 2/5/10 10:48 AM, "Corbin Hoenes" <corbin@tynt.com> wrote:
>>>
>>>> So I'm implementing a custom processor and my processor get's a recordEntry
>>>> passed to it's parse method.  It has more than a single log line it.  Is
>>>> this
>>>> the responsibility of my processor to split it up into lines?  The
>>>> nextLine()
>>>> method in AbstractProcessor made me think that I'd always be passed a single
>>>> line.
>>>>
>>>
>>
>
>



-- 
Ari Rabkin asrabkin@gmail.com
UC Berkeley Computer Science Department

Mime
View raw message