hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12596) bulkload needs to follow locality
Date Thu, 16 Jul 2015 08:54:06 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14629448#comment-14629448
] 

Lars Hofhansl commented on HBASE-12596:
---------------------------------------

Hmm... We usually do not add improvements to earlier branches but not into later branches,
So if it is in 0.98, we probably want this in 1.0, 1.1, and especially 1.2 as well.

[~apurtell], thoughts? Thing similar to this came up for 0.94 every now and then, where folks
complained either the earlier release "forcing" stuff into the later release, or alternatively
that later releases could veto changes into the earlier releases. Not sure how we want to
do this with 0.98, but the above is weird. Folks going from 0.98 to 1.0, 1.1, or 1.2 shouldn't
suddenly lose an improvement.

> bulkload needs to follow locality
> ---------------------------------
>
>                 Key: HBASE-12596
>                 URL: https://issues.apache.org/jira/browse/HBASE-12596
>             Project: HBase
>          Issue Type: Improvement
>          Components: HFile, regionserver
>    Affects Versions: 0.98.8
>         Environment: hadoop-2.3.0, hbase-0.98.8, jdk1.7
>            Reporter: Victor Xu
>            Assignee: Victor Xu
>             Fix For: 2.0.0, 0.98.14, 1.3.0
>
>         Attachments: HBASE-12596-0.98-v1.patch, HBASE-12596-0.98-v2.patch, HBASE-12596-0.98-v3.patch,
HBASE-12596-0.98-v4.patch, HBASE-12596-0.98-v5.patch, HBASE-12596-0.98-v6.patch, HBASE-12596-branch-1-v1.patch,
HBASE-12596-branch-1-v2.patch, HBASE-12596-master-v1.patch, HBASE-12596-master-v2.patch, HBASE-12596-master-v3.patch,
HBASE-12596-master-v4.patch, HBASE-12596-master-v5.patch, HBASE-12596-master-v6.patch, HBASE-12596.patch
>
>
> Normally, we have 2 steps to perform a bulkload: 1. use a job to write HFiles to be loaded;
2. Move these HFiles to the right hdfs directory. However, the locality could be loss during
the first step. Why not just write the HFiles directly into the right place? We can do this
easily because StoreFile.WriterBuilder has the "withFavoredNodes" method, and we just need
to call it in HFileOutputFormat's getNewWriter().
> This feature is enabled by default, and we could use 'hbase.bulkload.locality.sensitive.enabled=false'
to disable it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message