incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jerome Boulon <jbou...@netflix.com>
Subject Re: Chukwa Pig Data Passthrough
Date Mon, 04 Oct 2010 21:35:33 GMT
Hi Matt,
When I designed this, the schema was NOT available in Pig. I'm not sure if this has changed
or not.
So I'm using the constructor as a way to get around the lack of schema definition but if you
can get it now from the query & the storage handler then it should be a pretty easy thing
todo.
So do you know if the sql schema is now available in Pig?

/Jerome.

On 10/4/10 2:28 PM, "Matt Davies" <matt.davies@tynt.com> wrote:

Hey all-

Trying to do some operations utilizing Chukwa and Pig.  Would like to basically

1. Read in the data from HDFS
2. Do some SPLIT operations
3. write the various files out with all the fields as seen during the loading phase.


So, my question is this - is there a way to utilize the org.apache.hadoop.chukwa.ChukwaStorage();
engine to load in and then store out all the various fields without having to individually
define fields in the ChukwaStorage constructor?


Thanks,
Matt




Mime
View raw message