Return-Path: Delivered-To: apmail-hadoop-core-commits-archive@www.apache.org Received: (qmail 98589 invoked from network); 11 Mar 2009 21:27:56 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 11 Mar 2009 21:27:56 -0000 Received: (qmail 12384 invoked by uid 500); 11 Mar 2009 21:27:45 -0000 Delivered-To: apmail-hadoop-core-commits-archive@hadoop.apache.org Received: (qmail 12290 invoked by uid 500); 11 Mar 2009 21:27:45 -0000 Mailing-List: contact core-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-dev@hadoop.apache.org Delivered-To: mailing list core-commits@hadoop.apache.org Received: (qmail 12264 invoked by uid 99); 11 Mar 2009 21:27:44 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 Mar 2009 14:27:44 -0700 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 Mar 2009 21:27:44 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id 2680B2388920; Wed, 11 Mar 2009 21:27:21 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r752625 - in /hadoop/core/branches/branch-0.18: CHANGES.txt src/test/org/apache/hadoop/dfs/SimulatedFSDataset.java src/test/org/apache/hadoop/dfs/TestInjectionForSimulatedStorage.java Date: Wed, 11 Mar 2009 21:27:20 -0000 To: core-commits@hadoop.apache.org From: hairong@apache.org X-Mailer: svnmailer-1.0.8 Message-Id: <20090311212721.2680B2388920@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: hairong Date: Wed Mar 11 21:27:20 2009 New Revision: 752625 URL: http://svn.apache.org/viewvc?rev=752625&view=rev Log: Merge -r 752608:752609 to move the change of HADOOP-5412 from main to branch 0.18. Modified: hadoop/core/branches/branch-0.18/CHANGES.txt hadoop/core/branches/branch-0.18/src/test/org/apache/hadoop/dfs/SimulatedFSDataset.java hadoop/core/branches/branch-0.18/src/test/org/apache/hadoop/dfs/TestInjectionForSimulatedStorage.java Modified: hadoop/core/branches/branch-0.18/CHANGES.txt URL: http://svn.apache.org/viewvc/hadoop/core/branches/branch-0.18/CHANGES.txt?rev=752625&r1=752624&r2=752625&view=diff ============================================================================== --- hadoop/core/branches/branch-0.18/CHANGES.txt (original) +++ hadoop/core/branches/branch-0.18/CHANGES.txt Wed Mar 11 21:27:20 2009 @@ -13,6 +13,9 @@ HADOOP-5134. FSNamesystem#commitBlockSynchronization adds under-construction block locations to blocksMap. (Dhruba Borthakur via hairong) + HADOOP-5412. Simulated DataNode should not write to a block that's being + written by another thread. (hairong) + Release 0.18.3 - 2009-01-27 IMPROVEMENTS Modified: hadoop/core/branches/branch-0.18/src/test/org/apache/hadoop/dfs/SimulatedFSDataset.java URL: http://svn.apache.org/viewvc/hadoop/core/branches/branch-0.18/src/test/org/apache/hadoop/dfs/SimulatedFSDataset.java?rev=752625&r1=752624&r2=752625&view=diff ============================================================================== --- hadoop/core/branches/branch-0.18/src/test/org/apache/hadoop/dfs/SimulatedFSDataset.java (original) +++ hadoop/core/branches/branch-0.18/src/test/org/apache/hadoop/dfs/SimulatedFSDataset.java Wed Mar 11 21:27:20 2009 @@ -270,7 +270,9 @@ } public synchronized void unfinalizeBlock(Block b) throws IOException { - blockMap.remove(b); + if (isBeingWritten(b)) { + blockMap.remove(b); + } } public synchronized Block[] getBlockReport() { @@ -364,6 +366,15 @@ return binfo.isFinalized(); } + /* check if a block is created but not finalized */ + private synchronized boolean isBeingWritten(Block b) { + BInfo binfo = blockMap.get(b); + if (binfo == null) { + return false; + } + return !binfo.isFinalized(); + } + public String toString() { return getStorageInfo(); } @@ -375,6 +386,10 @@ throw new BlockAlreadyExistsException("Block " + b + " is valid, and cannot be written to."); } + if (isBeingWritten(b)) { + throw new BlockAlreadyExistsException("Block " + b + + " is being written, and cannot be written to."); + } BInfo binfo = new BInfo(b, true); blockMap.put(b, binfo); SimulatedOutputStream crcStream = new SimulatedOutputStream(); Modified: hadoop/core/branches/branch-0.18/src/test/org/apache/hadoop/dfs/TestInjectionForSimulatedStorage.java URL: http://svn.apache.org/viewvc/hadoop/core/branches/branch-0.18/src/test/org/apache/hadoop/dfs/TestInjectionForSimulatedStorage.java?rev=752625&r1=752624&r2=752625&view=diff ============================================================================== --- hadoop/core/branches/branch-0.18/src/test/org/apache/hadoop/dfs/TestInjectionForSimulatedStorage.java (original) +++ hadoop/core/branches/branch-0.18/src/test/org/apache/hadoop/dfs/TestInjectionForSimulatedStorage.java Wed Mar 11 21:27:20 2009 @@ -166,7 +166,7 @@ cluster = new MiniDFSCluster(0, conf, numDataNodes*2, false, true, null, null); - + cluster.waitActive(); Set uniqueBlocks = new HashSet(); for (int i=0; i