crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <>
Subject [jira] [Commented] (CRUNCH-212) Need target wrapper for HFileOuptutFormat
Date Mon, 10 Jun 2013 03:57:22 GMT


Josh Wills commented on CRUNCH-212:

Yeah-- it's like it would need to be its own library, like o.a.c.lib.hbase.BulkImporter or
BulkLoader or some such thing. You could make the target private to that lib so that folks
couldn't shoot themselves in the foot by using it directly, or you could create a public target
class and put all sorts of warnings and what not around it.
> Need target wrapper for HFileOuptutFormat
> -----------------------------------------
>                 Key: CRUNCH-212
>                 URL:
>             Project: Crunch
>          Issue Type: Improvement
>          Components: IO
>            Reporter: Chao Shi
> I need to import data to hbase from MR. I found HFileOutputFormat is ~5x more efficient
than HTableOutputFormat. So maybe we need a target wrapper for it.
> Future more, is it possible to call HBase to load it automatically after HFiles are generated?

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message