Return-Path: X-Original-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D671BBE6E for ; Fri, 7 Oct 2011 20:48:55 +0000 (UTC) Received: (qmail 39094 invoked by uid 500); 7 Oct 2011 20:48:55 -0000 Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org Received: (qmail 39060 invoked by uid 500); 7 Oct 2011 20:48:55 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-issues@hadoop.apache.org Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 39052 invoked by uid 99); 7 Oct 2011 20:48:55 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 07 Oct 2011 20:48:55 +0000 X-ASF-Spam-Status: No, hits=-2000.5 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.116] (HELO hel.zones.apache.org) (140.211.11.116) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 07 Oct 2011 20:48:51 +0000 Received: from hel.zones.apache.org (hel.zones.apache.org [140.211.11.116]) by hel.zones.apache.org (Postfix) with ESMTP id 2688B2AEAB6 for ; Fri, 7 Oct 2011 20:48:30 +0000 (UTC) Date: Fri, 7 Oct 2011 20:48:30 +0000 (UTC) From: "Todd Lipcon (Commented) (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: <410288721.9850.1318020510159.JavaMail.tomcat@hel.zones.apache.org> In-Reply-To: <1063398155.8205.1318000169934.JavaMail.tomcat@hel.zones.apache.org> Subject: [jira] [Commented] (HDFS-2414) TestDFSRollback fails intermittently MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HDFS-2414?page=3Dcom.atlassian.= jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D13123= 166#comment-13123166 ]=20 Todd Lipcon commented on HDFS-2414: ----------------------------------- I managed to reproduce the other failure too... I see this in one of the VE= RSION files: {code} CF=C2=BAE ^Utt<8c>=C3=A1=C3=83 1&:45:=C3=A02O=C3=BC=C3=98T <8d><8d>11=C3=9E= ^Y=C3=99mespacd"a=3D054a0=C3=873=C2=AF<8d>3 =C3=A6\u=C3=89t=C2=B7r^FD=3D^_=C2=BAstCl<98>st=C3=ABrI=C3=A2 cwime^S<99> sto:a=C3=86eTypr=C2=A8MA6E^DNOjEg_=C3=8Bo^[k^RoflID=3D=C2=A3P-15<94>32=C3= =8D073^Qk=C3=9A27.<82>.0=C3=86O-1Y+1020p=C3=B0IA<9f>=C3=AE Eayout<9f>er=C3=8C_<83>n=C2=B8-38o {code} wow... putting my thinking cap on here. =20 > TestDFSRollback fails intermittently > ------------------------------------ > > Key: HDFS-2414 > URL: https://issues.apache.org/jira/browse/HDFS-2414 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node, test > Affects Versions: 0.23.0 > Reporter: Robert Joseph Evans > Priority: Critical > Attachments: hdfs-2414.txt, run-106-failed.tgz, run-158-failed.tg= z > > > When running TestDFSRollback repeatedly in a loop I observed a failure ra= te of about 3%. Two separate stack traces are in the output and it appears= to have something to do with not writing out a complete snapshot of the da= ta for rollback. > {noformat} > -------------------------------------------------------------------------= ------ > Test set: org.apache.hadoop.hdfs.TestDFSRollback > -------------------------------------------------------------------------= ------ > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 8.514 sec= <<< FAILURE! > testRollback(org.apache.hadoop.hdfs.TestDFSRollback) Time elapsed: 8.34 = sec <<< FAILURE! > java.lang.AssertionError: File contents differed: > /home/evans/src/hadoop-git/hadoop-hdfs-project/hadoop-hdfs/target/test/= data/dfs/data2/current/VERSION=3D5b19197114fad0a254e3f318b7f14aec > /home/evans/src/hadoop-git/hadoop-hdfs-project/hadoop-hdfs/target/test/= data/dfs/data1/current/VERSION=3Dea7b000a6a1711169fc7a836b240a991 > at org.junit.Assert.fail(Assert.java:91) > at org.apache.hadoop.hdfs.server.namenode.FSImageTestUtil.assertF= ileContentsSame(FSImageTestUtil.java:250) > at org.apache.hadoop.hdfs.server.namenode.FSImageTestUtil.assertP= arallelFilesAreIdentical(FSImageTestUtil.java:236) > at org.apache.hadoop.hdfs.TestDFSRollback.checkResult(TestDFSRoll= back.java:86) > at org.apache.hadoop.hdfs.TestDFSRollback.testRollback(TestDFSRol= lback.java:171) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccess= orImpl.java:39) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMeth= odAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at junit.framework.TestCase.runTest(TestCase.java:168) > at junit.framework.TestCase.runBare(TestCase.java:134) > at junit.framework.TestResult$1.protect(TestResult.java:110) > at junit.framework.TestResult.runProtected(TestResult.java:128) > at junit.framework.TestResult.run(TestResult.java:113) > at junit.framework.TestCase.run(TestCase.java:124) > at junit.framework.TestSuite.runTest(TestSuite.java:232) > at junit.framework.TestSuite.run(TestSuite.java:227) > at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38Class= Runner.java:83) > at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4T= estSet.java:59) > at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.exe= cuteTestSet(AbstractDirectoryTestSuite.java:120) > at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.exe= cute(AbstractDirectoryTestSuite.java:145) > at org.apache.maven.surefire.Surefire.run(Surefire.java:104) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccess= orImpl.java:39) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMeth= odAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInPro= cess(SurefireBooter.java:290) > at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireB= ooter.java:1017) > {noformat} > is the more common one, but I also saw > {noformat} > -------------------------------------------------------------------------= ------ > Test set: org.apache.hadoop.hdfs.TestDFSRollback > -------------------------------------------------------------------------= ------ > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 7.471 sec= <<< FAILURE! > testRollback(org.apache.hadoop.hdfs.TestDFSRollback) Time elapsed: 7.304= sec <<< FAILURE! > junit.framework.AssertionFailedError: Expected substring 'file VERSION ha= s layoutVersion missing' in exception but got: java.lang.IllegalArgumentExc= eption: Malformed \uxxxx encoding. > at java.util.Properties.loadConvert(Properties.java:552) > at java.util.Properties.load0(Properties.java:374) > at java.util.Properties.load(Properties.java:325) > at org.apache.hadoop.hdfs.server.common.Storage.readPropertiesFil= e(Storage.java:837) > at org.apache.hadoop.hdfs.server.common.Storage.readPreviousVersi= onProperties(Storage.java:789) > at org.apache.hadoop.hdfs.server.namenode.FSImage.doRollback(FSIm= age.java:439) > at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransiti= onRead(FSImage.java:270) > at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage= (FSDirectory.java:174) > at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize= (FSNamesystem.java:294) > at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.(FSN= amesystem.java:266) > at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem= (NameNode.java:292) > at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(Nam= eNode.java:326) > at org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNod= e.java:452) > at org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNod= e.java:444) > at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode= (NameNode.java:742) > at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNode(MiniDFSCl= uster.java:637) > at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniD= FSCluster.java:541) > at org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.ja= va:257) > at org.apache.hadoop.hdfs.MiniDFSCluster.(MiniDFSCluster.ja= va:85) > at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSClu= ster.java:243) > at org.apache.hadoop.hdfs.TestDFSRollback.startNameNodeShouldFail= (TestDFSRollback.java:100) > at org.apache.hadoop.hdfs.TestDFSRollback.testRollback(TestDFSRol= lback.java:268) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccess= orImpl.java:39) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMeth= odAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at junit.framework.TestCase.runTest(TestCase.java:168) > at junit.framework.TestCase.runBare(TestCase.java:134) > at junit.framework.TestResult$1.protect(TestResult.java:110) > at junit.framework.TestResult.runProtected(TestResult.java:128) > at junit.framework.TestResult.run(TestResult.java:113) > at junit.framework.TestCase.run(TestCase.java:124) > at junit.framework.TestSuite.runTest(TestSuite.java:232) > at junit.framework.TestSuite.run(TestSuite.java:227) > at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38Class= Runner.java:83) > at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4T= estSet.java:59) > at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.exe= cuteTestSet(AbstractDirectoryTestSuite.java:120) > at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.exe= cute(AbstractDirectoryTestSuite.java:145) > at org.apache.maven.surefire.Surefire.run(Surefire.java:104) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccess= orImpl.java:39) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMeth= odAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInPro= cess(SurefireBooter.java:290) > at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireB= ooter.java:1017) > at junit.framework.Assert.fail(Assert.java:47) > at org.apache.hadoop.hdfs.TestDFSRollback.startNameNodeShouldFail= (TestDFSRollback.java:109) > at org.apache.hadoop.hdfs.TestDFSRollback.testRollback(TestDFSRol= lback.java:268) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccess= orImpl.java:39) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMeth= odAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at junit.framework.TestCase.runTest(TestCase.java:168) > at junit.framework.TestCase.runBare(TestCase.java:134) > at junit.framework.TestResult$1.protect(TestResult.java:110) > at junit.framework.TestResult.runProtected(TestResult.java:128) > at junit.framework.TestResult.run(TestResult.java:113) > at junit.framework.TestCase.run(TestCase.java:124) > at junit.framework.TestSuite.runTest(TestSuite.java:232) > at junit.framework.TestSuite.run(TestSuite.java:227) > at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38Class= Runner.java:83) > at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4T= estSet.java:59) > at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.exe= cuteTestSet(AbstractDirectoryTestSuite.java:120) > at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.exe= cute(AbstractDirectoryTestSuite.java:145) > at org.apache.maven.surefire.Surefire.run(Surefire.java:104) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccess= orImpl.java:39) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMeth= odAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInPro= cess(SurefireBooter.java:290) > at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireB= ooter.java:1017) > {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrato= rs: https://issues.apache.org/jira/secure/ContactAdministrators!default.jsp= a For more information on JIRA, see: http://www.atlassian.com/software/jira