chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gerrit van Vuuren <gvanvuu...@specificmedia.com>
Subject Re: Problem in ChukwaRecord file contents
Date Wed, 09 Jun 2010 16:24:25 GMT
I'm using this currently with the DirTailing adaptor and have multiple rows in each record.
It might be the Dir adaptor that does this, I haven't looked. 
------Original Message------
From: Corbin Hoenes
To: chukwa-user@hadoop.apache.org
Cc: 'Stuti Awasthi'
ReplyTo: chukwa-user@hadoop.apache.org
Subject: Re: Problem in ChukwaRecord file contents
Sent: 9 Jun 2010 16:00

Gerrit - It's my understanding that the CharFileTailingAdaptorUTF8 sends only 1 line per record.
 Why can't Stuti just use this? On Jun 8, 2010, at 7:55 AM, Gerrit Jansen van Vuuren wrote:
Have a look at:     org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.LWFTAdaptor
  protected int extractRecords(ChunkReceiver eq, long buffOffsetInFile,       byte[]
buf) throws InterruptedException {     org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8
protected int extractRecords(ChunkReceiver eq, long buffOffsetInFile,       byte[] buf)
throws InterruptedException {   If you need one line per record you’d have to write your
own adaptor. Maybe subclass CharFileTailingAdaptorUTF8       From: Stuti Awasthi [mailto:Stuti_Awasthi@persistent.co.in] 
Sent: Tuesday, June 08, 2010 1:31 PM To: chukwa-user@hadoop.apache.org; Gerrit van Vuuren
Subject: RE: Problem in ChukwaRecord file contents   So is  that means that we will always
have several lines of log data in the <body> tag of chukwa record?  Can you please
tell me where is that agent code that defines this.   I have read these ChukwaRecord through
Map Reduce and can read the original log lines. J   Stuti   From: Gerrit Jansen van Vuuren
[mailto:gvanvuuren@specificmedia.com]  Sent: Tuesday, June 08, 2010 5:53 PM To: chukwa-user@hadoop.apache.org
Subject: RE: Problem in ChukwaRecord file contents   Each chukwa record will contain several
lines of log data (depending on how the agent defines lines J ).   You can use the MapReduce
Jobs, HDFS or Pig
Mime
View raw message