Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 7 Sep 2011 19:15:09 +0000 (UTC)
From: "jiraposter@reviews.apache.org (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: 
 <171966524.27139.1315422909798.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HBASE-2195) Support cyclic replication
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13099231#comment-13099231 ] 

jiraposter@reviews.apache.org commented on HBASE-2195:
------------------------------------------------------


bq.  On 2011-09-07 18:51:35, Jean-Daniel Cryans wrote:
bq.  > http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/replication/TestMasterReplication.java, line 96
bq.  > <https://reviews.apache.org/r/1730/diff/2/?file=38159#file38159line96>
bq.  >
bq.  >     seems to me there's an opportunity for refactoring with TestReplication

Yes, although I think the two will move in different directions over time.
For example I want to add testing with three clusters to setup a cycle soon, and then this would all change.


- Lars


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1730/#review1797
-----------------------------------------------------------


On 2011-09-07 18:49:31, Lars Hofhansl wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/1730/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2011-09-07 18:49:31)
bq.  
bq.  
bq.  Review request for hbase, Ted Yu, Michael Stack, and Jean-Daniel Cryans.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  For Master <-> Master replication (as well as cycles > 2) a replication sink needs to keep track who is was the originator of a change and
bq.  (1) do not replicate this data back to the originator and
bq.  (2) pass that information on to the next link in a cycle > 2
bq.  
bq.  In order to do that, HlogKeys read from the WAL are tagged with the Cluster UUID at the ReplicationSource before the logs are shipped to the Sink. The sink writes the Cluster UUID into the WAL log (in HlogKey, so only once per edit). If the sink is also source for yet another sink the ClusterUUID is retained in the HLogKeys read from the local WAL and passed on to the sink.
bq.  
bq.  
bq.  This addresses bug HBASE-2195.
bq.      https://issues.apache.org/jira/browse/HBASE-2195
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/HConstants.java 1166212 
bq.    http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Delete.java 1166212 
bq.    http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Put.java 1166212 
bq.    http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java 1166212 
bq.    http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1166212 
bq.    http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 1166212 
bq.    http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogKey.java 1166212 
bq.    http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java 1166212 
bq.    http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSink.java 1166212 
bq.    http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java 1166212 
bq.    http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java 1166212 
bq.    http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogMethods.java 1166212 
bq.    http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java 1166212 
bq.    http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALActionsListener.java 1166212 
bq.    http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/replication/TestMasterReplication.java PRE-CREATION 
bq.    http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationSource.java 1166212 
bq.    http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSink.java 1166212 
bq.    http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java 1166212 
bq.  
bq.  Diff: https://reviews.apache.org/r/1730/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  All tests pass.
bq.  New test: TestMasterReplication (which tests Master <-> Master, but no cycles > 2, yet), also passes.
bq.  Manually tested a cycle between 3 clusters.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Lars
bq.  
bq.


> Support cyclic replication
> --------------------------
>
>                 Key: HBASE-2195
>                 URL: https://issues.apache.org/jira/browse/HBASE-2195
>             Project: HBase
>          Issue Type: Sub-task
>          Components: replication
>            Reporter: Jean-Daniel Cryans
>            Assignee: Lars Hofhansl
>         Attachments: 2195-v5.txt, 2195-v6.txt, 2195.txt
>
>
> We need to support cyclic replication by using the cluster id of each HlogKey and stop replicating when it goes back to the original cluster.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira