Return-Path: Delivered-To: apmail-hadoop-zookeeper-commits-archive@minotaur.apache.org Received: (qmail 61344 invoked from network); 5 May 2010 22:29:05 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 5 May 2010 22:29:05 -0000 Received: (qmail 4578 invoked by uid 500); 5 May 2010 22:29:05 -0000 Delivered-To: apmail-hadoop-zookeeper-commits-archive@hadoop.apache.org Received: (qmail 4552 invoked by uid 500); 5 May 2010 22:29:05 -0000 Mailing-List: contact zookeeper-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: zookeeper-dev@ Delivered-To: mailing list zookeeper-commits@hadoop.apache.org Received: (qmail 4544 invoked by uid 99); 5 May 2010 22:29:05 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 05 May 2010 22:29:05 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 05 May 2010 22:29:01 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id 8703323888FE; Wed, 5 May 2010 22:28:10 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r941520 - in /hadoop/zookeeper/branches/branch-3.3: ./ src/java/main/org/apache/zookeeper/server/quorum/ src/java/test/org/apache/zookeeper/test/ Date: Wed, 05 May 2010 22:28:10 -0000 To: zookeeper-commits@hadoop.apache.org From: mahadev@apache.org X-Mailer: svnmailer-1.0.8 Message-Id: <20100505222810.8703323888FE@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: mahadev Date: Wed May 5 22:28:09 2010 New Revision: 941520 URL: http://svn.apache.org/viewvc?rev=941520&view=rev Log: ZOOKEEPER-764. Observer elected leader due to inconsistent voting view (henry via mahadev) Modified: hadoop/zookeeper/branches/branch-3.3/CHANGES.txt hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/FastLeaderElection.java hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/LeaderElection.java hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeer.java hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeerMain.java hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/QuorumZooKeeperServer.java hadoop/zookeeper/branches/branch-3.3/src/java/test/org/apache/zookeeper/test/AsyncHammerTest.java hadoop/zookeeper/branches/branch-3.3/src/java/test/org/apache/zookeeper/test/HierarchicalQuorumTest.java hadoop/zookeeper/branches/branch-3.3/src/java/test/org/apache/zookeeper/test/LENonTerminateTest.java hadoop/zookeeper/branches/branch-3.3/src/java/test/org/apache/zookeeper/test/QuorumBase.java Modified: hadoop/zookeeper/branches/branch-3.3/CHANGES.txt URL: http://svn.apache.org/viewvc/hadoop/zookeeper/branches/branch-3.3/CHANGES.txt?rev=941520&r1=941519&r2=941520&view=diff ============================================================================== --- hadoop/zookeeper/branches/branch-3.3/CHANGES.txt (original) +++ hadoop/zookeeper/branches/branch-3.3/CHANGES.txt Wed May 5 22:28:09 2010 @@ -37,10 +37,13 @@ BUGFIXES: (Kapil Thangavelu via henryr) ZOOKEEPER-737. some 4 letter words may fail with netcat (nc) (mahadev) - + ZOOKEEPER-763. Deadlock on close w/ zkpython / c client (henry via phunt) + ZOOKEEPER-764. Observer elected leader due to inconsistent voting view + (henry via mahadev) + Release 3.3.0 - 2010-03-24 Non-backward compatible changes: Modified: hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/FastLeaderElection.java URL: http://svn.apache.org/viewvc/hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/FastLeaderElection.java?rev=941520&r1=941519&r2=941520&view=diff ============================================================================== --- hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/FastLeaderElection.java (original) +++ hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/FastLeaderElection.java Wed May 5 22:28:09 2010 @@ -587,7 +587,7 @@ public class FastLeaderElection implemen * @return ServerState */ private ServerState learningState(){ - if(self.getPeerType() == LearnerType.PARTICIPANT){ + if(self.getLearnerType() == LearnerType.PARTICIPANT){ LOG.debug("I'm a participant: " + self.getId()); return ServerState.FOLLOWING; } @@ -603,7 +603,7 @@ public class FastLeaderElection implemen * @return long */ private long getInitId(){ - if(self.getPeerType() == LearnerType.PARTICIPANT) + if(self.getLearnerType() == LearnerType.PARTICIPANT) return self.getId(); else return Long.MIN_VALUE; } @@ -614,7 +614,7 @@ public class FastLeaderElection implemen * @return long */ private long getInitLastLoggedZxid(){ - if(self.getPeerType() == LearnerType.PARTICIPANT) + if(self.getLearnerType() == LearnerType.PARTICIPANT) return self.getLastLoggedZxid(); else return Long.MIN_VALUE; } Modified: hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/LeaderElection.java URL: http://svn.apache.org/viewvc/hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/LeaderElection.java?rev=941520&r1=941519&r2=941520&view=diff ============================================================================== --- hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/LeaderElection.java (original) +++ hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/LeaderElection.java Wed May 5 22:28:09 2010 @@ -226,7 +226,7 @@ public class LeaderElection implements E self.setCurrentVote(result.winner); s.close(); Vote current = self.getCurrentVote(); - LOG.info("Found leader: my type is: " + self.getPeerType()); + LOG.info("Found leader: my type is: " + self.getLearnerType()); /* * We want to make sure we implement the state machine * correctly. If we are a PARTICIPANT, once a leader @@ -234,7 +234,7 @@ public class LeaderElection implements E * FOLLOWING. However if we are an OBSERVER, it is an * error to be elected as a Leader. */ - if (self.getPeerType() == LearnerType.OBSERVER) { + if (self.getLearnerType() == LearnerType.OBSERVER) { if (current.id == self.getId()) { // This should never happen! LOG.error("OBSERVER elected as leader!"); Modified: hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeer.java URL: http://svn.apache.org/viewvc/hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeer.java?rev=941520&r1=941519&r2=941520&view=diff ============================================================================== --- hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeer.java (original) +++ hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeer.java Wed May 5 22:28:09 2010 @@ -148,14 +148,24 @@ public class QuorumPeer extends Thread i /* * Default value of peer is participant */ - private LearnerType peerType = LearnerType.PARTICIPANT; + private LearnerType learnerType = LearnerType.PARTICIPANT; - public LearnerType getPeerType() { - return peerType; + public LearnerType getLearnerType() { + return learnerType; } - public void setPeerType(LearnerType p) { - peerType = p; + /** + * Sets the LearnerType both in the QuorumPeer and in the peerMap + */ + public void setLearnerType(LearnerType p) { + learnerType = p; + if (quorumPeers.containsKey(this.myid)) { + this.quorumPeers.get(myid).type = p; + } else { + LOG.error("Setting LearnerType to " + p + " but " + myid + + " not in QuorumPeers. "); + } + } /** * The servers that make up the cluster Modified: hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeerMain.java URL: http://svn.apache.org/viewvc/hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeerMain.java?rev=941520&r1=941519&r2=941520&view=diff ============================================================================== --- hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeerMain.java (original) +++ hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeerMain.java Wed May 5 22:28:09 2010 @@ -138,7 +138,7 @@ public class QuorumPeerMain { quorumPeer.setQuorumVerifier(config.getQuorumVerifier()); quorumPeer.setCnxnFactory(cnxnFactory); quorumPeer.setZKDatabase(new ZKDatabase(quorumPeer.getTxnFactory())); - quorumPeer.setPeerType(config.getPeerType()); + quorumPeer.setLearnerType(config.getPeerType()); quorumPeer.start(); quorumPeer.join(); Modified: hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/QuorumZooKeeperServer.java URL: http://svn.apache.org/viewvc/hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/QuorumZooKeeperServer.java?rev=941520&r1=941519&r2=941520&view=diff ============================================================================== --- hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/QuorumZooKeeperServer.java (original) +++ hadoop/zookeeper/branches/branch-3.3/src/java/main/org/apache/zookeeper/server/quorum/QuorumZooKeeperServer.java Wed May 5 22:28:09 2010 @@ -57,6 +57,6 @@ public abstract class QuorumZooKeeperSer pwriter.print("quorumPort="); pwriter.println(self.quorumPeers.get(self.getId()).addr.getPort()); pwriter.print("peerType="); - pwriter.println(self.getPeerType().ordinal()); + pwriter.println(self.getLearnerType().ordinal()); } } Modified: hadoop/zookeeper/branches/branch-3.3/src/java/test/org/apache/zookeeper/test/AsyncHammerTest.java URL: http://svn.apache.org/viewvc/hadoop/zookeeper/branches/branch-3.3/src/java/test/org/apache/zookeeper/test/AsyncHammerTest.java?rev=941520&r1=941519&r2=941520&view=diff ============================================================================== --- hadoop/zookeeper/branches/branch-3.3/src/java/test/org/apache/zookeeper/test/AsyncHammerTest.java (original) +++ hadoop/zookeeper/branches/branch-3.3/src/java/test/org/apache/zookeeper/test/AsyncHammerTest.java Wed May 5 22:28:09 2010 @@ -23,8 +23,6 @@ import static org.apache.zookeeper.test. import java.util.LinkedList; -import junit.framework.TestCase; - import org.apache.log4j.Logger; import org.apache.zookeeper.CreateMode; import org.apache.zookeeper.KeeperException; @@ -36,11 +34,10 @@ import org.apache.zookeeper.AsyncCallbac import org.apache.zookeeper.ZooDefs.Ids; import org.apache.zookeeper.data.Stat; import org.apache.zookeeper.test.ClientBase.CountdownWatcher; -import org.junit.After; -import org.junit.Before; +import org.junit.Assert; import org.junit.Test; -public class AsyncHammerTest extends TestCase +public class AsyncHammerTest implements StringCallback, VoidCallback, DataCallback { private static final Logger LOG = Logger.getLogger(AsyncHammerTest.class); @@ -49,15 +46,11 @@ public class AsyncHammerTest extends Tes private volatile boolean bang; - @Before - @Override - protected void setUp() throws Exception { - LOG.info("STARTING " + getName()); - qb.setUp(); + public void setUp(boolean withObservers) throws Exception { + qb.setUp(withObservers); } protected void restart() throws Exception { - LOG.info("RESTARTING " + getName()); qb.tearDown(); // don't call setup - we don't want to reassign ports/dirs, etc... @@ -65,12 +58,9 @@ public class AsyncHammerTest extends Tes qb.startServers(); } - @After - @Override - protected void tearDown() throws Exception { + public void tearDown() throws Exception { LOG.info("Test clients shutting down"); qb.tearDown(); - LOG.info("FINISHED " + getName()); } /** @@ -130,8 +120,8 @@ public class AsyncHammerTest extends Tes } private synchronized void decOutstanding() { - outstanding--; - assertTrue("outstanding >= 0", outstanding >= 0); + outstanding--; + Assert.assertTrue("outstanding >= 0", outstanding >= 0); notifyAll(); } @@ -175,6 +165,7 @@ public class AsyncHammerTest extends Tes @Test public void testHammer() throws Exception { + setUp(false); bang = true; LOG.info("Starting hammers"); HammerThread[] hammers = new HammerThread[100]; @@ -189,7 +180,7 @@ public class AsyncHammerTest extends Tes for (int i = 0; i < hammers.length; i++) { hammers[i].interrupt(); verifyThreadTerminated(hammers[i], 60000); - assertFalse(hammers[i].failed); + Assert.assertFalse(hammers[i].failed); } // before restart @@ -201,12 +192,12 @@ public class AsyncHammerTest extends Tes // after restart LOG.info("Verifying hammers 2"); qb.verifyRootOfAllServersMatch(qb.hostPort); + tearDown(); } @Test public void testObserversHammer() throws Exception { - qb.tearDown(); - qb.setUp(true); + setUp(true); bang = true; Thread[] hammers = new Thread[100]; for (int i = 0; i < hammers.length; i++) { @@ -220,7 +211,8 @@ public class AsyncHammerTest extends Tes verifyThreadTerminated(hammers[i], 60000); } // before restart - qb.verifyRootOfAllServersMatch(qb.hostPort); + qb.verifyRootOfAllServersMatch(qb.hostPort); + tearDown(); } @SuppressWarnings("unchecked") Modified: hadoop/zookeeper/branches/branch-3.3/src/java/test/org/apache/zookeeper/test/HierarchicalQuorumTest.java URL: http://svn.apache.org/viewvc/hadoop/zookeeper/branches/branch-3.3/src/java/test/org/apache/zookeeper/test/HierarchicalQuorumTest.java?rev=941520&r1=941519&r2=941520&view=diff ============================================================================== --- hadoop/zookeeper/branches/branch-3.3/src/java/test/org/apache/zookeeper/test/HierarchicalQuorumTest.java (original) +++ hadoop/zookeeper/branches/branch-3.3/src/java/test/org/apache/zookeeper/test/HierarchicalQuorumTest.java Wed May 5 22:28:09 2010 @@ -176,7 +176,7 @@ public class HierarchicalQuorumTest exte QuorumHierarchical hq4 = new QuorumHierarchical(qp); s4 = new QuorumPeer(peers, s4dir, s4dir, port4, 3, 4, tickTime, initLimit, syncLimit, hq4); if (withObservers) { - s4.setPeerType(QuorumPeer.LearnerType.OBSERVER); + s4.setLearnerType(QuorumPeer.LearnerType.OBSERVER); } assertEquals(port4, s4.getClientPort()); @@ -184,7 +184,7 @@ public class HierarchicalQuorumTest exte QuorumHierarchical hq5 = new QuorumHierarchical(qp); s5 = new QuorumPeer(peers, s5dir, s5dir, port5, 3, 5, tickTime, initLimit, syncLimit, hq5); if (withObservers) { - s5.setPeerType(QuorumPeer.LearnerType.OBSERVER); + s5.setLearnerType(QuorumPeer.LearnerType.OBSERVER); } assertEquals(port5, s5.getClientPort()); Modified: hadoop/zookeeper/branches/branch-3.3/src/java/test/org/apache/zookeeper/test/LENonTerminateTest.java URL: http://svn.apache.org/viewvc/hadoop/zookeeper/branches/branch-3.3/src/java/test/org/apache/zookeeper/test/LENonTerminateTest.java?rev=941520&r1=941519&r2=941520&view=diff ============================================================================== --- hadoop/zookeeper/branches/branch-3.3/src/java/test/org/apache/zookeeper/test/LENonTerminateTest.java (original) +++ hadoop/zookeeper/branches/branch-3.3/src/java/test/org/apache/zookeeper/test/LENonTerminateTest.java Wed May 5 22:28:09 2010 @@ -171,7 +171,7 @@ public class LENonTerminateTest extends self.setCurrentVote(result.winner); s.close(); Vote current = self.getCurrentVote(); - LOG.info("Found leader: my type is: " + self.getPeerType()); + LOG.info("Found leader: my type is: " + self.getLearnerType()); /* * We want to make sure we implement the state machine * correctly. If we are a PARTICIPANT, once a leader @@ -179,7 +179,7 @@ public class LENonTerminateTest extends * FOLLOWING. However if we are an OBSERVER, it is an * error to be elected as a Leader. */ - if (self.getPeerType() == LearnerType.OBSERVER) { + if (self.getLearnerType() == LearnerType.OBSERVER) { if (current.id == self.getId()) { // This should never happen! LOG.error("OBSERVER elected as leader!"); Modified: hadoop/zookeeper/branches/branch-3.3/src/java/test/org/apache/zookeeper/test/QuorumBase.java URL: http://svn.apache.org/viewvc/hadoop/zookeeper/branches/branch-3.3/src/java/test/org/apache/zookeeper/test/QuorumBase.java?rev=941520&r1=941519&r2=941520&view=diff ============================================================================== --- hadoop/zookeeper/branches/branch-3.3/src/java/test/org/apache/zookeeper/test/QuorumBase.java (original) +++ hadoop/zookeeper/branches/branch-3.3/src/java/test/org/apache/zookeeper/test/QuorumBase.java Wed May 5 22:28:09 2010 @@ -159,10 +159,16 @@ public class QuorumBase extends ClientBa assertEquals(port5, s5.getClientPort()); if (withObservers) { - s4.setPeerType(LearnerType.OBSERVER); - s5.setPeerType(LearnerType.OBSERVER); + s4.setLearnerType(LearnerType.OBSERVER); + s5.setLearnerType(LearnerType.OBSERVER); } + LOG.info("QuorumPeer 1 voting view: " + s1.getVotingView()); + LOG.info("QuorumPeer 2 voting view: " + s2.getVotingView()); + LOG.info("QuorumPeer 3 voting view: " + s3.getVotingView()); + LOG.info("QuorumPeer 4 voting view: " + s4.getVotingView()); + LOG.info("QuorumPeer 5 voting view: " + s5.getVotingView()); + LOG.info("start QuorumPeer 1"); s1.start(); LOG.info("start QuorumPeer 2");