From hdfs-commits-return-4233-apmail-hadoop-hdfs-commits-archive=hadoop.apache.org@hadoop.apache.org Fri Aug 10 03:47:59 2012 Return-Path: X-Original-To: apmail-hadoop-hdfs-commits-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7A03ED39F for ; Fri, 10 Aug 2012 03:47:59 +0000 (UTC) Received: (qmail 2960 invoked by uid 500); 10 Aug 2012 03:47:59 -0000 Delivered-To: apmail-hadoop-hdfs-commits-archive@hadoop.apache.org Received: (qmail 2869 invoked by uid 500); 10 Aug 2012 03:47:58 -0000 Mailing-List: contact hdfs-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-dev@hadoop.apache.org Delivered-To: mailing list hdfs-commits@hadoop.apache.org Received: (qmail 2856 invoked by uid 99); 10 Aug 2012 03:47:58 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 10 Aug 2012 03:47:58 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 10 Aug 2012 03:47:56 +0000 Received: from eris.apache.org (localhost [127.0.0.1]) by eris.apache.org (Postfix) with ESMTP id 4648723888EA; Fri, 10 Aug 2012 03:47:13 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1371556 - in /hadoop/common/branches/branch-2/hadoop-hdfs-project: ./ hadoop-hdfs/ hadoop-hdfs/src/main/java/ hadoop-hdfs/src/main/native/ hadoop-hdfs/src/main/native/fuse-dfs/test/ hadoop-hdfs/src/main/webapps/datanode/ hadoop-hdfs/src/ma... Date: Fri, 10 Aug 2012 03:47:12 -0000 To: hdfs-commits@hadoop.apache.org From: eli@apache.org X-Mailer: svnmailer-1.0.8-patched Message-Id: <20120810034713.4648723888EA@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: eli Date: Fri Aug 10 03:47:12 2012 New Revision: 1371556 URL: http://svn.apache.org/viewvc?rev=1371556&view=rev Log: HDFS-3758. TestFuseDFS test failing. Contributed by Colin Patrick McCabe Modified: hadoop/common/branches/branch-2/hadoop-hdfs-project/ (props changed) hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/ (props changed) hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/src/main/java/ (props changed) hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/src/main/native/ (props changed) hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/src/main/native/fuse-dfs/test/TestFuseDFS.java hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode/ (props changed) hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/ (props changed) hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary/ (props changed) hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs/ (props changed) Propchange: hadoop/common/branches/branch-2/hadoop-hdfs-project/ ------------------------------------------------------------------------------ Merged /hadoop/common/trunk/hadoop-hdfs-project:r1371555 Propchange: hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/ ------------------------------------------------------------------------------ Merged /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs:r1371555 Modified: hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt?rev=1371556&r1=1371555&r2=1371556&view=diff ============================================================================== --- hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt (original) +++ hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Fri Aug 10 03:47:12 2012 @@ -424,6 +424,8 @@ Release 2.0.1-alpha - UNRELEASED HDFS-3721. hsync support broke wire compatibility. (todd and atm) + HDFS-3758. TestFuseDFS test failing. (Colin Patrick McCabe via eli) + BREAKDOWN OF HDFS-3042 SUBTASKS HDFS-2185. HDFS portion of ZK-based FailoverController (todd) Propchange: hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/src/main/java/ ------------------------------------------------------------------------------ Merged /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java:r1371555 Propchange: hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/src/main/native/ ------------------------------------------------------------------------------ Merged /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/native:r1371555 Modified: hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/src/main/native/fuse-dfs/test/TestFuseDFS.java URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/src/main/native/fuse-dfs/test/TestFuseDFS.java?rev=1371556&r1=1371555&r2=1371556&view=diff ============================================================================== --- hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/src/main/native/fuse-dfs/test/TestFuseDFS.java (original) +++ hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/src/main/native/fuse-dfs/test/TestFuseDFS.java Fri Aug 10 03:47:12 2012 @@ -44,6 +44,7 @@ public class TestFuseDFS { private static MiniDFSCluster cluster; private static FileSystem fs; + private static Process fuseProcess; private static Runtime r; private static String mountPoint; @@ -137,8 +138,28 @@ public class TestFuseDFS { assertEquals("File content differs", expectedContents, s); } + private static class RedirectToStdoutThread extends Thread { + private InputStream is; + + RedirectToStdoutThread(InputStream is) { + this.is = is; + } + public void run() { + try { + InputStreamReader isr = new InputStreamReader(is); + BufferedReader br = new BufferedReader(isr); + String line=null; + while ( (line = br.readLine()) != null) { + LOG.error("FUSE_LINE:" + line); + } + } catch (IOException e) { + e.printStackTrace(); + } + } + } + /** Run a fuse-dfs process to mount the given DFS */ - private static void establishMount(URI uri) throws IOException { + private static Process establishMount(URI uri) throws IOException { Runtime r = Runtime.getRuntime(); String cp = System.getProperty("java.class.path"); @@ -163,6 +184,8 @@ public class TestFuseDFS { "-obig_writes", // Allow >4kb writes "-oentry_timeout=0.1", // Don't cache dents long "-oattribute_timeout=0.1", // Don't cache attributes long + "-ononempty", // Don't complain about junk in mount point + "-f", // Don't background the process "-ordbuffer=32768", // Read buffer size in kb "rw" }; @@ -178,17 +201,35 @@ public class TestFuseDFS { execAssertSucceeds("mkdir -p " + mountPoint); // Mount the mini cluster + String cmdStr = ""; + for (String c : mountCmd) { + cmdStr += (" " + c); + } + LOG.info("now mounting with:" + cmdStr); + Process fuseProcess = r.exec(mountCmd, env); + RedirectToStdoutThread stdoutThread = + new RedirectToStdoutThread(fuseProcess.getInputStream()); + RedirectToStdoutThread stderrThread = + new RedirectToStdoutThread(fuseProcess.getErrorStream()); + stdoutThread.start(); + stderrThread.start(); + // Wait for fusermount to start up, so that we know we're operating on the + // FUSE FS when we run the tests. try { - Process fuseProcess = r.exec(mountCmd, env); - assertEquals(0, fuseProcess.waitFor()); - } catch (InterruptedException ie) { - fail("Failed to mount"); + Thread.sleep(50000); + } catch (InterruptedException e) { } + return fuseProcess; } /** Tear down the fuse-dfs process and mount */ private static void teardownMount() throws IOException { execWaitRet("fusermount -u " + mountPoint); + try { + assertEquals(0, fuseProcess.waitFor()); // fuse_dfs should exit cleanly + } catch (InterruptedException e) { + fail("interrupted while waiting for fuse_dfs process to exit."); + } } @BeforeClass @@ -200,7 +241,7 @@ public class TestFuseDFS { cluster = new MiniDFSCluster.Builder(conf).build(); cluster.waitClusterUp(); fs = cluster.getFileSystem(); - establishMount(fs.getUri()); + fuseProcess = establishMount(fs.getUri()); } @AfterClass Propchange: hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode/ ------------------------------------------------------------------------------ Merged /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode:r1371555 Propchange: hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/ ------------------------------------------------------------------------------ Merged /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs:r1371555 Propchange: hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary/ ------------------------------------------------------------------------------ Merged /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary:r1371555 Propchange: hadoop/common/branches/branch-2/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs/ ------------------------------------------------------------------------------ Merged /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/hdfs:r1371555