crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Micah Whitacre (JIRA)" <>
Subject [jira] [Commented] (CRUNCH-212) Need target wrapper for HFileOuptutFormat
Date Thu, 01 Aug 2013 02:29:49 GMT


Micah Whitacre commented on CRUNCH-212:

Haven't gotten to test out the patch but excited about the progress.  Do have a few comments:

* Should document justification in the HFileOutFormatForCrunch class on why we are using that
over HFileOutputFormat.
* Lines 63 & 74 of the code you retrieve the exact same path. (Path path = getDefaultWorkFile(context,
> Need target wrapper for HFileOuptutFormat
> -----------------------------------------
>                 Key: CRUNCH-212
>                 URL:
>             Project: Crunch
>          Issue Type: Improvement
>          Components: IO
>            Reporter: Chao Shi
>         Attachments: crunch-212-draft2.patch, crunch-212-draft.patch, crunch-212-v0.patch
> I need to import data to hbase from MR. I found HFileOutputFormat is ~5x more efficient
than HTableOutputFormat. So maybe we need a target wrapper for it.
> Future more, is it possible to call HBase to load it automatically after HFiles are generated?

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message