incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stuti Awasthi <Stuti_Awas...@persistent.co.in>
Subject RE: Problem in ChukwaRecord file contents
Date Fri, 11 Jun 2010 05:13:16 GMT
Hey Guys, 

Thanks a lot. Problem is solved by CharFileTailingAdapterUTF8. Now each record is handled
per line of Chukwa Records. 

Thanks
Cheers!!

-----Original Message-----
From: Gerrit van Vuuren [mailto:gvanvuuren@specificmedia.com] 
Sent: Friday, June 11, 2010 12:18 AM
To: 'chukwa-user@hadoop.apache.org'
Subject: Re: Problem in ChukwaRecord file contents

Thanks,

----- Original Message -----
From: Ariel Rabkin <asrabkin@gmail.com>
To: chukwa-user@hadoop.apache.org <chukwa-user@hadoop.apache.org>
Sent: Thu Jun 10 19:45:05 2010
Subject: Re: Problem in ChukwaRecord file contents

Howdy --

The .done files hold a sequence of Chunks. Each chunk will include
several lines, plus a table of record endpoints, indicating where the
linebreaks are.

The demux job, not the collector, uses this line break table in order
to output a set of ChukwaRecords, one line per record.

--Ari

On Thu, Jun 10, 2010 at 11:40 AM, Gerrit van Vuuren
<gvanvuuren@specificmedia.com> wrote:
> Hi,
>
> Ok then this is a bug.
>
> I have some general questions:
> Does this mean that each record in the done file written by the collector will only contain
one line? Or will the collector batch these up?
>
> Cheers,
> ------Original Message------
> From: Stuti Awasthi
> To: chukwa-user@hadoop.apache.org
> Cc: asrabkin@gmail.com
> Cc: corbin@tynt.com
> Cc: Gerrit van Vuuren
> Subject: RE: Problem in ChukwaRecord file contents
> Sent: 10 Jun 2010 06:02
>
> Currently I am using FileTailing Adapter. I will try with CharFileTailingAdaptorUTF8.
>
> -----Original Message-----
> From: Ariel Rabkin [mailto:asrabkin@gmail.com]
> Sent: Thursday, June 10, 2010 3:13 AM
> To: chukwa-user@hadoop.apache.org
> Subject: Re: Problem in ChukwaRecord file contents
>
> Yes, CharFileTailingAdaptorUTF8 is designed to do one line per record.
> If it does more than that, it's a bug.
>
> --Ari
>
> On Wed, Jun 9, 2010 at 8:00 AM, Corbin Hoenes <corbin@tynt.com> wrote:
>> Gerrit -
>> It's my understanding that the CharFileTailingAdaptorUTF8 sends only 1 line
>> per record.  Why can't Stuti just use this?
>> On Jun 8, 2010, at 7:55 AM, Gerrit Jansen van Vuuren wrote:
>>
>> Have a look at:
>>
>>
>> org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.LWFTAdaptor
>>
>> protected int extractRecords(ChunkReceiver eq, long buffOffsetInFile,
>>       byte[] buf) throws InterruptedException {
>>
>>
>> org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8
>> protected int extractRecords(ChunkReceiver eq, long buffOffsetInFile,
>>       byte[] buf) throws InterruptedException {
>>
>> If you need one line per record you'd have to write your own adaptor. Maybe
>> subclass CharFileTailingAdaptorUTF8
>>
>>
>>
>> From: Stuti Awasthi [mailto:Stuti_Awasthi@persistent.co.in]
>> Sent: Tuesday, June 08, 2010 1:31 PM
>> To: chukwa-user@hadoop.apache.org; Gerrit van Vuuren
>> Subject: RE: Problem in ChukwaRecord file contents
>>
>> So is  that means
>
> ------Original Message Truncated------
>



-- 
Ari Rabkin asrabkin@gmail.com
UC Berkeley Computer Science Department

DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Persistent
Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed.
If you are not the intended recipient, you are not authorized to read, retain, copy, print,
distribute or use this message. If you have received this communication in error, please notify
the sender and delete all copies of this message. Persistent Systems Ltd. does not accept
any liability for virus infected mails.

Mime
View raw message