incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <>
Subject Re: Custom Processor
Date Thu, 11 Feb 2010 23:52:59 GMT
I am ok with this.  Ari, comment?


On 2/11/10 3:49 PM, "Corbin Hoenes" <> wrote:

> This maybe a bit of a nit pick but what about instead of calling this
> nextLine() it was called nextRecord()?
> On Feb 5, 2010, at 12:57 PM, Eric Yang wrote:
>> Log output by ChukwaLog4J appender has a special property that it will group
>> the same log output statement delimited by ^A character.  Hence, the UTF8
>> adaptor will not break up the structures and send the entire output as one
>> record.  When the custom processor get an recordEntry (part of a chunk), it
>> will contain the full output.  nextLine function is to get the next record
>> shipped in the same chunk.  It should not be rely upon, unless the specially
>> crafted adaptor is shipping multi-records as a chunk and this chunk has no
>> dependency outside.  The current UTF8 adaptor is sending multiple records
>> depending on the speed of the tail is happening.  UTF8 adaptor should not be
>> rely upon if you want to use nextLine feature to build your parser states.
>> Regards,
>> Eric
>> On 2/5/10 10:48 AM, "Corbin Hoenes" <> wrote:
>>> So I'm implementing a custom processor and my processor get's a recordEntry
>>> passed to it's parse method.  It has more than a single log line it.  Is
>>> this
>>> the responsibility of my processor to split it up into lines?  The
>>> nextLine()
>>> method in AbstractProcessor made me think that I'd always be passed a single
>>> line.

View raw message