chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bill Graham (JIRA)" <>
Subject [jira] Updated: (CHUKWA-495) Implement Pig 0.7.0 compatible Loader and Storer classes
Date Mon, 19 Jul 2010 22:15:49 GMT


Bill Graham updated CHUKWA-495:

    Attachment: CHUKWA-495.1.patch

Attached is CHUKWA-495.1.patch, which includes the new classes mentioned above, along with
a refactored {{ChukwaArchive}} class. Please review these changes. Note the change in line
82 of {{TestLocalChukwaStorage}}. Basically, when setting the output path is set to {{chukwa-pig.evt}},
Pig writes out to {{chukwa-pig.evt/part-m-00000}}. I'm not sure how to keep that from happening
so we need to make sure that change is ok.

I've also attached pig-0.7.0 jars built from the pig distro at this mirror (pig.jar and pig-test.jar
should be removed):

I wasn't able to find a pig distro in the maven repositories listed in the maven configs.
If we find one, we can handle that change in a seperate JIRA.

Finally, I don't know what the purpose of having contrib/chukwa-pig/chukwa-pig.jar in SVN,
since it's generated at build time but I haven't removed it. I've uploaded my version too,
in the event we need to commit that to what's current.

> Implement Pig 0.7.0 compatible Loader and Storer classes
> --------------------------------------------------------
>                 Key: CHUKWA-495
>                 URL:
>             Project: Chukwa
>          Issue Type: Improvement
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>         Attachments: CHUKWA-495.1.patch, chukwa-pig.jar, pig-0.7.0-test.jar, pig-0.7.0.jar.gz
> Pig 0.7.0 introduces a revamped Load/Store model that is not backward compatible with
previous Pig releases. We need to create new classes to handle loading/storing Chukwa data
from Pig. Since the new load/store model uses abstract super classes instead of interfaces,
I propose we deprecate {{org.apache.hadoop.chukwa.ChukwaStorage}} and create the following
> {noformat}
> org.apache.hadoop.chukwa.pig.ChukwaLoader
> org.apache.hadoop.chukwa.pig.ChukwaStorer
> {noformat}
> Note the addition of the pig sub-package. Thoughts about this approach?

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message