incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stuti Awasthi <Stuti_Awas...@persistent.co.in>
Subject RE: Why ChukwaRecord have only Key:Value as Strings
Date Mon, 26 Jul 2010 05:28:27 GMT
Thanks Eric,

I have few queries .

1)      How will Avro be used with Chukwa?

2)      Does all Chukwa files be in Avro format?

3)      Are there any plans to enhance Chukwa record format?


I have written Adapter and Parser for Multiline Record format. If Chukwa will be using Avro
format then I also have to change my code.
Currently I am processing the log files in Chukwa and converting them to Avro format to keep
it in HDFS. If you are planning to include the Avro in the Chukwa then does it mean that all
the Chukwa files will be in Avro format ?

Please Suggest
Stuti

From: Eric Yang [mailto:eyang@yahoo-inc.com]
Sent: Friday, July 23, 2010 10:06 PM
To: chukwa-user@hadoop.apache.org
Cc: Jerome Boulon
Subject: Re: Why ChukwaRecord have only Key:Value as Strings

Initially, ChukwaRecord only supports String because it was made to process text log file.
 We were naïve to think that we can use JSON for all our data.  There is a plan to use Avro
instead of supporting generic types when Avro mapreduce input/output format is ready next
month.  This provides better meta data support inside the data for the processing system.

Regards,
Eric

On 7/23/10 5:03 AM, "Stuti Awasthi" <Stuti_Awasthi@persistent.co.in> wrote:
Hi all,

I was looking at the code of ChukwaRecord and  found out that it adds only <String key
& String value >

Snippet :

Public class ChukwaRecord extends ChukwaRecordJT implements Record
{
Public void add( String Key, String Value )
}

I have a scenario in which I want to add the Object as a value i.e <String Key ,Object
value> .

Does chukwa's current implementation support that or any patch available?

Stuti


Thanks and Regards

Stuti Awasthi | Software Engineer - IBM BU | Persistent Systems Limited
stuti_awasthi@persistent.co.in <mailto:chandan_avdhut@persistent.co.in> |  Tel: +91
(20) 391 77837


DISCLAIMER ========== This e-mail may contain privileged and confidential information which
is the property of Persistent Systems Ltd. It is intended only for the use of the individual
or entity to which it is addressed. If you are not the intended recipient, you are not authorized
to read, retain, copy, print, distribute or use this message. If you have received this communication
in error, please notify the sender and delete all copies of this message. Persistent Systems
Ltd. does not accept any liability for virus infected mails.

DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Persistent
Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed.
If you are not the intended recipient, you are not authorized to read, retain, copy, print,
distribute or use this message. If you have received this communication in error, please notify
the sender and delete all copies of this message. Persistent Systems Ltd. does not accept
any liability for virus infected mails.

Mime
View raw message