hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dave Latham (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12596) bulkload needs to follow locality
Date Thu, 16 Jul 2015 16:18:06 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14629951#comment-14629951
] 

Dave Latham commented on HBASE-12596:
-------------------------------------

Andrew's note that 0.98 is like major, 0.98.x is like minor, 0.98.x.y is like patch (not 0.98
minor, 0.98.x patch) is illuminating.  It does show a couple differences in the processes
though.  By that definition, before 1.0 there were RMs for major versions (after 1.0 there
are RMs for each minor version); before 1.0 patch releases were very rare and only happened
for the most recent minor release (after 1.0 it looks like the intent would be to have patch
releases for multiple minor releases living on).  I think it reflects an intent for making
more stable releases available by updating minor releases with patches rather than just focusing
on the latest minor release and leaving other ones behind.

> bulkload needs to follow locality
> ---------------------------------
>
>                 Key: HBASE-12596
>                 URL: https://issues.apache.org/jira/browse/HBASE-12596
>             Project: HBase
>          Issue Type: Improvement
>          Components: HFile, regionserver
>    Affects Versions: 0.98.8
>         Environment: hadoop-2.3.0, hbase-0.98.8, jdk1.7
>            Reporter: Victor Xu
>            Assignee: Victor Xu
>             Fix For: 2.0.0, 0.98.14, 1.3.0
>
>         Attachments: HBASE-12596-0.98-v1.patch, HBASE-12596-0.98-v2.patch, HBASE-12596-0.98-v3.patch,
HBASE-12596-0.98-v4.patch, HBASE-12596-0.98-v5.patch, HBASE-12596-0.98-v6.patch, HBASE-12596-branch-1-v1.patch,
HBASE-12596-branch-1-v2.patch, HBASE-12596-master-v1.patch, HBASE-12596-master-v2.patch, HBASE-12596-master-v3.patch,
HBASE-12596-master-v4.patch, HBASE-12596-master-v5.patch, HBASE-12596-master-v6.patch, HBASE-12596.patch
>
>
> Normally, we have 2 steps to perform a bulkload: 1. use a job to write HFiles to be loaded;
2. Move these HFiles to the right hdfs directory. However, the locality could be loss during
the first step. Why not just write the HFiles directly into the right place? We can do this
easily because StoreFile.WriterBuilder has the "withFavoredNodes" method, and we just need
to call it in HFileOutputFormat's getNewWriter().
> This feature is enabled by default, and we could use 'hbase.bulkload.locality.sensitive.enabled=false'
to disable it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message