hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "HBase Review Board (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2223) Handle 10min+ network partitions between clusters
Date Fri, 11 Jun 2010 22:34:14 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12878113#action_12878113
] 

HBase Review Board commented on HBASE-2223:
-------------------------------------------

Message from: stack@duboce.net

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/76/#review194
-----------------------------------------------------------


First pass.  Maybe by the 3rd pass I'll have an idea of whats going on.  General comment is
that there is a lot of new code here but tests seem to test replication system.  There are
few instances of unit tests ensuring newly added methods are working properly.


src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java
<http://review.hbase.org/r/76/#comment894>

    For sure setConf will have been called before we get here?  So, stuff gets setup by setConf?
 Can setConf be called more than once?  How do I know how to use this class?  Not doc'd. 
Doesn't have a Constructor.



src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java
<http://review.hbase.org/r/76/#comment895>

    The way this is done, if I didn't want to wait on the ttl, then I'd have to write a new
class.  Can't we have ttl and recplication be distinct and then if I want delete based off
ttl and whether log up in zk, then chain them?



src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSink.java
<http://review.hbase.org/r/76/#comment896>

    I dont follow?



src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSink.java
<http://review.hbase.org/r/76/#comment897>

    Should read this out of config. rather than hardcode 10.



src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSink.java
<http://review.hbase.org/r/76/#comment898>

    Same here.



src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSink.java
<http://review.hbase.org/r/76/#comment900>

    Long while loop; can break it up?



src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSink.java
<http://review.hbase.org/r/76/#comment899>

    Only operate on the first kv?



src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSink.java
<http://review.hbase.org/r/76/#comment901>

    Do you have to write position back to zk?



src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSink.java
<http://review.hbase.org/r/76/#comment903>

    Can code from HLog be used here?



src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
<http://review.hbase.org/r/76/#comment904>

    This ain't a constructor?



src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
<http://review.hbase.org/r/76/#comment907>

    We have to copy?



src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java
<http://review.hbase.org/r/76/#comment908>

    Not a constructor.  If javadoc in an interface, you don't need to reproduce the javadoc
in the implementation.



src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java
<http://review.hbase.org/r/76/#comment909>

    This should be SortedSet, not TreeSet... or NavigableSet.



src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
<http://review.hbase.org/r/76/#comment910>

    Good



src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java
<http://review.hbase.org/r/76/#comment911>

    What does this class do?



src/test/java/org/apache/hadoop/hbase/replication/TestReplicationSource.java
<http://review.hbase.org/r/76/#comment912>

    No dfs in this test.  Thats intentional?



src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSink.java
<http://review.hbase.org/r/76/#comment913>

    Can't you squash some of these tests together?  They each start up own minidfscluster...
just start it once?


- stack





> Handle 10min+ network partitions between clusters
> -------------------------------------------------
>
>                 Key: HBASE-2223
>                 URL: https://issues.apache.org/jira/browse/HBASE-2223
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Jean-Daniel Cryans
>            Assignee: Jean-Daniel Cryans
>             Fix For: 0.21.0
>
>         Attachments: HBASE-2223.patch
>
>
> We need a nice way of handling long network partitions without impacting a master cluster
(which pushes the data). Currently it will just retry over and over again.
> I think we could:
>  - Stop replication to a slave cluster if it didn't respond for more than 10 minutes
>  - Keep track of the duration of the partition
>  - When the slave cluster comes back, initiate a MR job like HBASE-2221 
> Maybe we want less than 10 minutes, maybe we want this to be all automatic or just the
first 2 parts. Discuss.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message