hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hsieh (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4650) Update LoadIncrementalHFiles to use atomic bulk load RS mechanism
Date Mon, 24 Oct 2011 17:13:32 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13134253#comment-13134253
] 

Jonathan Hsieh commented on HBASE-4650:
---------------------------------------

I'm in the process of cleaning up the modifications to LoadIncrementalHFiles and adding more
tests before submitting HBASE-4650 for a for review.  This cut passes the two tests that use
LoadIncrementalHFiles (TestLoadIncrementalHFiles and TestHFileOutputFormat).  I'll post a
preliminary version for those interested.

In the code, there are significant logic changes due to grouping so I've chosen to take out
the concurrency on the first cut because gathering and splitting HFiles into proper groups
introduces a synchronization point that prevents some of the concurrency as before.  This
is because groups need to be fully gathered before bulk loads in a region is attempted.  I'll
include comments where concurrency is ok.  

Before I spend effort to parallelize this implementation more, I want to add another test
to verify that this works while splits are going on.

                
> Update LoadIncrementalHFiles to use atomic bulk load RS mechanism
> -----------------------------------------------------------------
>
>                 Key: HBASE-4650
>                 URL: https://issues.apache.org/jira/browse/HBASE-4650
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver
>            Reporter: Jonathan Hsieh
>             Fix For: 0.92.0
>
>
> MR jobs and command line bulk load execution runs use the LoadIncrementalHFile.doBulkLoad.
 This needs to be updated to group HFiles by row/region so that rows can be atomically loaded
multiple column families.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message