Mailing-List: contact chukwa-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: chukwa-user@hadoop.apache.org
Received-SPF: softfail (athena.apache.org: transitioning domain of
 corbin@tynt.com does not designate 64.71.238.89 as permitted sender)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Apple Message framework v1077)
Subject: Re: Custom Processor
From: Corbin Hoenes <corbin@tynt.com>
In-Reply-To: <C791B829.642E%eyang@yahoo-inc.com>
Date: Fri, 5 Feb 2010 14:47:35 -0700
Content-Transfer-Encoding: quoted-printable
Message-Id: <8B90078D-0BD9-4143-9F91-A999CDC15584@tynt.com>
References: <C791B829.642E%eyang@yahoo-inc.com>
To: chukwa-user@hadoop.apache.org

Thanks Eric... things are looking much more sane now after understanding =
what a recordEntry is.  :)

On Feb 5, 2010, at 12:57 PM, Eric Yang wrote:

> Log output by ChukwaLog4J appender has a special property that it will =
group
> the same log output statement delimited by ^A character.  Hence, the =
UTF8
> adaptor will not break up the structures and send the entire output as =
one
> record.  When the custom processor get an recordEntry (part of a =
chunk), it
> will contain the full output.  nextLine function is to get the next =
record
> shipped in the same chunk.  It should not be rely upon, unless the =
specially
> crafted adaptor is shipping multi-records as a chunk and this chunk =
has no
> dependency outside.  The current UTF8 adaptor is sending multiple =
records
> depending on the speed of the tail is happening.  UTF8 adaptor should =
not be
> rely upon if you want to use nextLine feature to build your parser =
states.
>=20
> Regards,
> Eric
>=20
> On 2/5/10 10:48 AM, "Corbin Hoenes" <corbin@tynt.com> wrote:
>=20
>> So I'm implementing a custom processor and my processor get's a =
recordEntry
>> passed to it's parse method.  It has more than a single log line it.  =
Is this
>> the responsibility of my processor to split it up into lines?  The =
nextLine()
>> method in AbstractProcessor made me think that I'd always be passed a =
single
>> line.
>>=20
>=20