hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashish Singhi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13153) Bulk Loaded HFile Replication
Date Sat, 07 Nov 2015 17:52:11 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14995324#comment-14995324
] 

Ashish Singhi commented on HBASE-13153:
---------------------------------------

Thanks for the comments [~tedyu]

bq. 1. secure bulk loading (without replication)
There is no change in flow of secure bulk load without replication. We just added a check
if the input hfile path and staging dir hfile path are same avoid FS rename. As in replication
the staging dir is managed by it and all the hfiles are already copied in it so we save this
FS rename call.

{quote}
2. bulk loaded hfiles replicated across secure clusters
3. 2. bulk loaded hfiles replicated across secure HA clusters
{quote}
As for secure clusters we need to configure kerberos settings required for below operations
across two secure clusters
1. HDFS distcp and
2. Existing HBase replication(mutations).
These settings users taking bulk load data backup may be already configuring it in their secure
clusters env. Nothing additional for this feature.

> Bulk Loaded HFile Replication
> -----------------------------
>
>                 Key: HBASE-13153
>                 URL: https://issues.apache.org/jira/browse/HBASE-13153
>             Project: HBase
>          Issue Type: New Feature
>          Components: Replication
>            Reporter: sunhaitao
>            Assignee: Ashish Singhi
>             Fix For: 2.0.0
>
>         Attachments: HBASE-13153-v1.patch, HBASE-13153-v10.patch, HBASE-13153-v11.patch,
HBASE-13153-v12.patch, HBASE-13153-v2.patch, HBASE-13153-v3.patch, HBASE-13153-v4.patch, HBASE-13153-v5.patch,
HBASE-13153-v6.patch, HBASE-13153-v7.patch, HBASE-13153-v8.patch, HBASE-13153-v9.patch, HBASE-13153.patch,
HBase Bulk Load Replication-v1-1.pdf, HBase Bulk Load Replication-v2.pdf, HBase Bulk Load
Replication-v3.pdf, HBase Bulk Load Replication.pdf, HDFS_HA_Solution.PNG
>
>
> Currently we plan to use HBase Replication feature to deal with disaster tolerance scenario.But
we encounter an issue that we will use bulkload very frequently,because bulkload bypass write
path, and will not generate WAL, so the data will not be replicated to backup cluster. It's
inappropriate to bukload twice both on active cluster and backup cluster. So i advise do some
modification to bulkload feature to enable bukload to both active cluster and backup cluster



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message