crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Micah Whitacre (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CRUNCH-212) Need target wrapper for HFileOuptutFormat
Date Wed, 17 Jul 2013 03:06:49 GMT

    [ https://issues.apache.org/jira/browse/CRUNCH-212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13710646#comment-13710646
] 

Micah Whitacre commented on CRUNCH-212:
---------------------------------------

Reading through the discussion on this bug there seems to be two different pieces of functionality.

1. Ability to create HFiles
2. Ability to load generated HFiles into HBase.

We typically perform the loading as a separate workflow to handle various operation concerns.
 So not forcing the loading as a requirement of generating the HFiles would be ideal.
                
> Need target wrapper for HFileOuptutFormat
> -----------------------------------------
>
>                 Key: CRUNCH-212
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-212
>             Project: Crunch
>          Issue Type: Improvement
>          Components: IO
>            Reporter: Chao Shi
>
> I need to import data to hbase from MR. I found HFileOutputFormat is ~5x more efficient
than HTableOutputFormat. So maybe we need a target wrapper for it.
> Future more, is it possible to call HBase to load it automatically after HFiles are generated?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message