hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashish Singhi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13153) Bulk Loaded HFile Replication
Date Mon, 23 Nov 2015 08:38:11 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15021743#comment-15021743
] 

Ashish Singhi commented on HBASE-13153:
---------------------------------------

[~jerryhe], thanks for the comments.

bq. Another region server RPC handler --> holds region write lock --> transfers files
to be bulk loaded into the region from remote cluster synchronously
Not remote cluster, it will be local at this point as all the files are copied first from
source to peer cluster and then only bulk load is initiated which will just rename these files.

bq. Multiple handlers on the peer cluster can potentially be blocked
Yes agress, this point was earlier raised by [~devaraj] also I have noted it down and have
plan to may be add another QoS for bulk load as part of another jira as it will also help
in normal bulk load case.

bq. Now that the peer cluster 'server id' needs to read files directly from source cluster
hbase.root directory. In a secure cluster, I recall that the hbase.root has been changed to
be only accessible by the current 'server id'. Now they need to match
I did not get what you mean. But we have done internal testing for this by providing peer
cluster user read permission on the source cluster FS as mentioned in the design doc.

> Bulk Loaded HFile Replication
> -----------------------------
>
>                 Key: HBASE-13153
>                 URL: https://issues.apache.org/jira/browse/HBASE-13153
>             Project: HBase
>          Issue Type: New Feature
>          Components: Replication
>            Reporter: sunhaitao
>            Assignee: Ashish Singhi
>             Fix For: 2.0.0
>
>         Attachments: HBASE-13153-v1.patch, HBASE-13153-v10.patch, HBASE-13153-v11.patch,
HBASE-13153-v12.patch, HBASE-13153-v13.patch, HBASE-13153-v14.patch, HBASE-13153-v15.patch,
HBASE-13153-v16.patch, HBASE-13153-v17.patch, HBASE-13153-v18.patch, HBASE-13153-v2.patch,
HBASE-13153-v3.patch, HBASE-13153-v4.patch, HBASE-13153-v5.patch, HBASE-13153-v6.patch, HBASE-13153-v7.patch,
HBASE-13153-v8.patch, HBASE-13153-v9.patch, HBASE-13153.patch, HBase Bulk Load Replication-v1-1.pdf,
HBase Bulk Load Replication-v2.pdf, HBase Bulk Load Replication-v3.pdf, HBase Bulk Load Replication.pdf,
HDFS_HA_Solution.PNG
>
>
> Currently we plan to use HBase Replication feature to deal with disaster tolerance scenario.But
we encounter an issue that we will use bulkload very frequently,because bulkload bypass write
path, and will not generate WAL, so the data will not be replicated to backup cluster. It's
inappropriate to bukload twice both on active cluster and backup cluster. So i advise do some
modification to bulkload feature to enable bukload to both active cluster and backup cluster



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message