hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "santosh banerjee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-9888) HBase replicates edits written before the replication peer is created
Date Wed, 06 Nov 2013 21:32:19 GMT

    [ https://issues.apache.org/jira/browse/HBASE-9888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815313#comment-13815313
] 

santosh banerjee commented on HBASE-9888:
-----------------------------------------

{quote} In 0.94, HLogKey has a writeTime and we could seek in the current WAL until we find
an edit that's been written after the source was created.{quote}

This sounds interesting. So,are you suggesting implementing a custom ReplicationSource to
seek into the current WAL to find the edit with writeTime > sourceCreationTimestamp?


> HBase replicates edits written before the replication peer is created
> ---------------------------------------------------------------------
>
>                 Key: HBASE-9888
>                 URL: https://issues.apache.org/jira/browse/HBASE-9888
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Dave Latham
>
> When creating a new replication peer the ReplicationSourceManager enqueues the currently
open HLog to the ReplicationSource to ship to the destination cluster.  The ReplicationSource
starts at the beginning of the HLog and ships over any pre-existing writes.
> A workaround is to roll all the HLogs before enabling replication.
> A little background for how it affected us - we were migrating one cluster in a master-master
pair.  I.e. transitioning from A <\-> B to B <-> C.  After shutting down writes
from A -> B we enabled writes from C -> B.  However, this replicated some earlier writes
that were in C's HLogs that had originated in A.  Since we were running a version of HBase
before HBASE-7709 those writes then got caught in a infinite replication cycle and bringing
down region servers OOM because of HBASE-9865.
> However, in general, if one wants to manage what data gets replicated, one wouldn't expect
that potentially very old writes would be included when setting up a new replication link.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message