hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12596) bulkload needs to follow locality
Date Wed, 15 Jul 2015 03:31:05 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14627462#comment-14627462
] 

Hudson commented on HBASE-12596:
--------------------------------

FAILURE: Integrated in HBase-0.98 #1057 (See [https://builds.apache.org/job/HBase-0.98/1057/])
Revert "HBASE-12596 bulkload needs to follow locality (Victor Xu)" (apurtell: rev 3094ecb26611f93fe4bebe45c211d2eb3590f09b)
* hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat2.java
HBASE-12596 bulkload needs to follow locality (Victor Xu) (apurtell: rev 0c86068f8879551db643dc5ac67610e7bdd29bb6)
* hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat2.java
* hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java


> bulkload needs to follow locality
> ---------------------------------
>
>                 Key: HBASE-12596
>                 URL: https://issues.apache.org/jira/browse/HBASE-12596
>             Project: HBase
>          Issue Type: Improvement
>          Components: HFile, regionserver
>    Affects Versions: 0.98.8
>         Environment: hadoop-2.3.0, hbase-0.98.8, jdk1.7
>            Reporter: Victor Xu
>            Assignee: Victor Xu
>             Fix For: 2.0.0, 0.98.14, 1.3.0
>
>         Attachments: HBASE-12596-0.98-v1.patch, HBASE-12596-0.98-v2.patch, HBASE-12596-0.98-v3.patch,
HBASE-12596-0.98-v4.patch, HBASE-12596-0.98-v5.patch, HBASE-12596-0.98-v6.patch, HBASE-12596-branch-1-v1.patch,
HBASE-12596-branch-1-v2.patch, HBASE-12596-master-v1.patch, HBASE-12596-master-v2.patch, HBASE-12596-master-v3.patch,
HBASE-12596-master-v4.patch, HBASE-12596-master-v5.patch, HBASE-12596-master-v6.patch, HBASE-12596.patch
>
>
> Normally, we have 2 steps to perform a bulkload: 1. use a job to write HFiles to be loaded;
2. Move these HFiles to the right hdfs directory. However, the locality could be loss during
the first step. Why not just write the HFiles directly into the right place? We can do this
easily because StoreFile.WriterBuilder has the "withFavoredNodes" method, and we just need
to call it in HFileOutputFormat's getNewWriter().
> This feature is enabled by default, and we could use 'hbase.bulkload.locality.sensitive.enabled=false'
to disable it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message