Return-Path: Delivered-To: apmail-hadoop-core-commits-archive@www.apache.org Received: (qmail 46035 invoked from network); 17 Mar 2009 19:58:20 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 17 Mar 2009 19:58:20 -0000 Received: (qmail 18214 invoked by uid 500); 17 Mar 2009 19:58:20 -0000 Delivered-To: apmail-hadoop-core-commits-archive@hadoop.apache.org Received: (qmail 18183 invoked by uid 500); 17 Mar 2009 19:58:20 -0000 Mailing-List: contact core-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-dev@hadoop.apache.org Delivered-To: mailing list core-commits@hadoop.apache.org Received: (qmail 18174 invoked by uid 99); 17 Mar 2009 19:58:20 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 Mar 2009 12:58:20 -0700 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 Mar 2009 19:58:17 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id 9D11923888E7; Tue, 17 Mar 2009 19:57:56 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r755372 - in /hadoop/core/branches/branch-0.20: ./ CHANGES.txt src/hdfs/org/apache/hadoop/hdfs/server/namenode/DatanodeDescriptor.java src/hdfs/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java Date: Tue, 17 Mar 2009 19:57:56 -0000 To: core-commits@hadoop.apache.org From: hairong@apache.org X-Mailer: svnmailer-1.0.8 Message-Id: <20090317195756.9D11923888E7@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: hairong Date: Tue Mar 17 19:57:55 2009 New Revision: 755372 URL: http://svn.apache.org/viewvc?rev=755372&view=rev Log: Merge -r 755369:755370 to move the change of HADOOP-5479 from main to branch 0.20. Modified: hadoop/core/branches/branch-0.20/ (props changed) hadoop/core/branches/branch-0.20/CHANGES.txt (contents, props changed) hadoop/core/branches/branch-0.20/src/hdfs/org/apache/hadoop/hdfs/server/namenode/DatanodeDescriptor.java hadoop/core/branches/branch-0.20/src/hdfs/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java Propchange: hadoop/core/branches/branch-0.20/ ------------------------------------------------------------------------------ --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Mar 17 19:57:55 2009 @@ -1,2 +1,2 @@ /hadoop/core/branches/branch-0.19:713112 -/hadoop/core/trunk:727001,727117,727191,727212,727217,727228,727255,727869,728187,729052,729987,732385,732572,732613,732777,732838,732869,733887,734870,734916,736426,738328,738697,740077,740157,741703,741762,743745,743816,743892,744894,745180,746010,746206,746227,746233,746274,746338,746902-746903,746925,746944,746968,746970,747279,747289,747802,748084,748090,748783,749262,749318,749863,750533,752073,752609,752834,752836,752913,752932,753112-753113,753346,754645,754847,754927,755035,755226,755348 +/hadoop/core/trunk:727001,727117,727191,727212,727217,727228,727255,727869,728187,729052,729987,732385,732572,732613,732777,732838,732869,733887,734870,734916,736426,738328,738697,740077,740157,741703,741762,743745,743816,743892,744894,745180,746010,746206,746227,746233,746274,746338,746902-746903,746925,746944,746968,746970,747279,747289,747802,748084,748090,748783,749262,749318,749863,750533,752073,752609,752834,752836,752913,752932,753112-753113,753346,754645,754847,754927,755035,755226,755348,755370 Modified: hadoop/core/branches/branch-0.20/CHANGES.txt URL: http://svn.apache.org/viewvc/hadoop/core/branches/branch-0.20/CHANGES.txt?rev=755372&r1=755371&r2=755372&view=diff ============================================================================== --- hadoop/core/branches/branch-0.20/CHANGES.txt (original) +++ hadoop/core/branches/branch-0.20/CHANGES.txt Tue Mar 17 19:57:55 2009 @@ -794,6 +794,9 @@ HADOOP-5449. Fixes the history cleaner thread. (Amareshwari Sriramadasu via ddas) + HADOOP-5479. NameNode should not send empty block replication request to + DataNode. (hairong) + Release 0.19.1 - Unreleased IMPROVEMENTS Propchange: hadoop/core/branches/branch-0.20/CHANGES.txt ------------------------------------------------------------------------------ --- svn:mergeinfo (original) +++ svn:mergeinfo Tue Mar 17 19:57:55 2009 @@ -1,3 +1,3 @@ /hadoop/core/branches/branch-0.18/CHANGES.txt:727226 /hadoop/core/branches/branch-0.19/CHANGES.txt:713112 -/hadoop/core/trunk/CHANGES.txt:727001,727117,727191,727212,727228,727255,727869,728187,729052,729987,732385,732572,732613,732777,732838,732869,733887,734870,734916,735082,736426,738602,738697,739416,740077,740157,741703,741762,743296,743745,743816,743892,744894,745180,745268,746010,746193,746206,746227,746233,746274,746902-746903,746925,746944,746968,746970,747279,747289,747802,748084,748090,748783,749262,749318,749863,750533,752073,752514,752555,752590,752609,752834,752836,752913,752932,753112-753113,753346,754645,754847,754927,755035,755226,755348 +/hadoop/core/trunk/CHANGES.txt:727001,727117,727191,727212,727228,727255,727869,728187,729052,729987,732385,732572,732613,732777,732838,732869,733887,734870,734916,735082,736426,738602,738697,739416,740077,740157,741703,741762,743296,743745,743816,743892,744894,745180,745268,746010,746193,746206,746227,746233,746274,746902-746903,746925,746944,746968,746970,747279,747289,747802,748084,748090,748783,749262,749318,749863,750533,752073,752514,752555,752590,752609,752834,752836,752913,752932,753112-753113,753346,754645,754847,754927,755035,755226,755348,755370 Modified: hadoop/core/branches/branch-0.20/src/hdfs/org/apache/hadoop/hdfs/server/namenode/DatanodeDescriptor.java URL: http://svn.apache.org/viewvc/hadoop/core/branches/branch-0.20/src/hdfs/org/apache/hadoop/hdfs/server/namenode/DatanodeDescriptor.java?rev=755372&r1=755371&r2=755372&view=diff ============================================================================== --- hadoop/core/branches/branch-0.20/src/hdfs/org/apache/hadoop/hdfs/server/namenode/DatanodeDescriptor.java (original) +++ hadoop/core/branches/branch-0.20/src/hdfs/org/apache/hadoop/hdfs/server/namenode/DatanodeDescriptor.java Tue Mar 17 19:57:55 2009 @@ -68,20 +68,16 @@ } /** Dequeue */ - synchronized List poll(int numTargets) { - if (numTargets <= 0 || blockq.isEmpty()) { + synchronized List poll(int numBlocks) { + if (numBlocks <= 0 || blockq.isEmpty()) { return null; } - else { - List results = new ArrayList(); - for(; !blockq.isEmpty() && numTargets > 0; ) { - numTargets -= blockq.peek().targets.length; - if (numTargets >= 0) { - results.add(blockq.poll()); - } - } - return results; + + List results = new ArrayList(); + for(; !blockq.isEmpty() && numBlocks > 0; numBlocks--) { + results.add(blockq.poll()); } + return results; } } Modified: hadoop/core/branches/branch-0.20/src/hdfs/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java URL: http://svn.apache.org/viewvc/hadoop/core/branches/branch-0.20/src/hdfs/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java?rev=755372&r1=755371&r2=755372&view=diff ============================================================================== --- hadoop/core/branches/branch-0.20/src/hdfs/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java (original) +++ hadoop/core/branches/branch-0.20/src/hdfs/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java Tue Mar 17 19:57:55 2009 @@ -2437,12 +2437,8 @@ } // choose replication targets - int maxTargets = - maxReplicationStreams - srcNode.getNumberOfBlocksToBeReplicated(); - assert maxTargets > 0 : "Datanode " + srcNode.getName() - + " should have not been selected as a source for replication."; DatanodeDescriptor targets[] = replicator.chooseTarget( - Math.min(requiredReplication - numEffectiveReplicas, maxTargets), + requiredReplication - numEffectiveReplicas, srcNode, containingNodes, null, block.getNumBytes()); if(targets.length == 0) continue; @@ -2457,13 +2453,15 @@ // Move the block-replication into a "pending" state. // The reason we use 'pending' is so we can retry // replications that fail after an appropriate amount of time. + pendingReplications.add(block, targets.length); + NameNode.stateChangeLog.debug( + "BLOCK* block " + block + + " is moved from neededReplications to pendingReplications"); + + // remove from neededReplications if(numEffectiveReplicas + targets.length >= requiredReplication) { neededReplicationsIterator.remove(); // remove from neededReplications replIndex--; - pendingReplications.add(block, targets.length); - NameNode.stateChangeLog.debug( - "BLOCK* block " + block - + " is moved from neededReplications to pendingReplications"); } if (NameNode.stateChangeLog.isInfoEnabled()) { StringBuffer targetList = new StringBuffer("datanode(s)");