hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13153) enable bulkload to support replication
Date Tue, 01 Sep 2015 10:58:46 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14725180#comment-14725180

ramkrishna.s.vasudevan commented on HBASE-13153:

Nice doc Ashish.
bq.Replication module will be one of the BulkLoadActionsListener, 
Can you explain more on this?  This will get registered as how the WalActionsListener get
registered right?
bq.Plan is we will have our own implementation of BaseLogCleanerDelegate#getDeletableFiles
to ensure this.
True.  Good point.
One thing to note is that if you are not allowing the hfiles to be archived that is due to
compaction will it not be removed from the block cache also?  Till the replication is done
we will have it in the block cache also?  See to this point.  In a real case this file may
not be actually referred but it may be in the block cache for some time till it gets evicted.
I need to see this code if really this is happening. If so no problem. May be a small prototype
on this area of the feature will help you when going in for the full impl.

> enable bulkload to support replication
> --------------------------------------
>                 Key: HBASE-13153
>                 URL: https://issues.apache.org/jira/browse/HBASE-13153
>             Project: HBase
>          Issue Type: New Feature
>          Components: Replication
>            Reporter: sunhaitao
>            Assignee: Ashish Singhi
>             Fix For: 2.0.0
>         Attachments: HBase Bulk Load Replication.pdf
> Currently we plan to use HBase Replication feature to deal with disaster tolerance scenario.But
we encounter an issue that we will use bulkload very frequently,because bulkload bypass write
path, and will not generate WAL, so the data will not be replicated to backup cluster. It's
inappropriate to bukload twice both on active cluster and backup cluster. So i advise do some
modification to bulkload feature to enable bukload to both active cluster and backup cluster

This message was sent by Atlassian JIRA

View raw message