hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Malaska (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-10983) Enhance LoadIncrementalHFile with option to do splitting in a distributed fashon
Date Tue, 15 Apr 2014 13:34:18 GMT
Ted Malaska created HBASE-10983:
-----------------------------------

             Summary: Enhance LoadIncrementalHFile with option to do splitting in a distributed
fashon
                 Key: HBASE-10983
                 URL: https://issues.apache.org/jira/browse/HBASE-10983
             Project: HBase
          Issue Type: Improvement
            Reporter: Ted Malaska
            Priority: Minor


Currently LoadIncrementalHFile supports splitting HFiles if they don't match up with the current
regions of the table being imported too.  

However this functionality of reading and rewriting the HFile is done through a single JVM,
which limits the overall speed of the splitting process.

This jira will allow the user to set a flag or a threshold (on the total size of the HFiles
to be split) that may trigger the splitting logic to be executed through a Map Only job as
opposed to the existing thread pool in a single JVM. 

I will have the following goals when writing this patch:
1. Extend LoadIncrementalHFile
2. Reuse as much code from LoadIncrementalHFile as possible



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message