hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "churro morales (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-11409) Add more flexibility for input directory structure to LoadIncrementalHFiles
Date Wed, 25 Jun 2014 00:15:26 GMT
churro morales created HBASE-11409:
--------------------------------------

             Summary: Add more flexibility for input directory structure to LoadIncrementalHFiles
                 Key: HBASE-11409
                 URL: https://issues.apache.org/jira/browse/HBASE-11409
             Project: HBase
          Issue Type: Bug
    Affects Versions: 0.94.20
            Reporter: churro morales


Use case:

We were trying to combine two very large tables into a single table.  Thus we ran jobs in
one datacenter that populated certain column families and another datacenter which populated
other column families.  Took a snapshot and exported them to their respective datacenters.
 Wanted to simply take the hdfs restored snapshot and use LoadIncremental to merge the data.
 

It would be nice to add support where we could run LoadIncremental on a directory where the
depth of store files is something other than two (current behavior).  

With snapshots it would be nice if you could pass a restored hdfs snapshot's directory and
have the tool run.  

I am attaching a patch where I parameterize the bulkLoad timeout as well as the default store
file depth.  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message