hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Victor Xu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-12596) bulkload needs to follow locality
Date Thu, 27 Nov 2014 23:01:12 GMT
Victor Xu created HBASE-12596:

             Summary: bulkload needs to follow locality
                 Key: HBASE-12596
                 URL: https://issues.apache.org/jira/browse/HBASE-12596
             Project: HBase
          Issue Type: Improvement
          Components: HFile, regionserver
    Affects Versions: 0.98.8
         Environment: hadoop-2.3.0, hbase-0.98.8, jdk1.7
            Reporter: Victor Xu

Normally, we have 2 steps to perform a bulkload: 1. use a job to write HFiles to be loaded;
2. Move these HFiles to the right hdfs directory. However, the locality could be loss during
the first step. Why not just write the HFiles directly into the right place? We can do this
easily because StoreFile.WriterBuilder has the "withFavoredNodes" method, and we just need
to call it in HFileOutputFormat's getNewWriter().

This message was sent by Atlassian JIRA

View raw message