incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <ey...@yahoo-inc.com>
Subject Re: Custom Processor
Date Fri, 05 Feb 2010 19:57:29 GMT
Log output by ChukwaLog4J appender has a special property that it will group
the same log output statement delimited by ^A character.  Hence, the UTF8
adaptor will not break up the structures and send the entire output as one
record.  When the custom processor get an recordEntry (part of a chunk), it
will contain the full output.  nextLine function is to get the next record
shipped in the same chunk.  It should not be rely upon, unless the specially
crafted adaptor is shipping multi-records as a chunk and this chunk has no
dependency outside.  The current UTF8 adaptor is sending multiple records
depending on the speed of the tail is happening.  UTF8 adaptor should not be
rely upon if you want to use nextLine feature to build your parser states.

Regards,
Eric

On 2/5/10 10:48 AM, "Corbin Hoenes" <corbin@tynt.com> wrote:

> So I'm implementing a custom processor and my processor get's a recordEntry
> passed to it's parse method.  It has more than a single log line it.  Is this
> the responsibility of my processor to split it up into lines?  The nextLine()
> method in AbstractProcessor made me think that I'd always be passed a single
> line.
> 


Mime
View raw message