Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Subject: Re: test failures - newbie help needed
MIME-Version: 1.0
From: Steve Loughran <stevel@hortonworks.com>
To: Simon Scott <Simon.Scott@viavisolutions.com>
CC: "common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>
Thread-Topic: test failures - newbie help needed
Thread-Index: AdKeQIGTP/ygScN5SwuR9srcFYSOCQAR5KwA
Date: Thu, 16 Mar 2017 12:04:08 +0000
Message-ID: <32D2DFF5-484C-4A40-AD14-AEA8B349182A@hortonworks.com>
References: <BY2PR18MB029692D26E757F38AF7A44F299260@BY2PR18MB0296.namprd18.prod.outlook.com>
In-Reply-To: <BY2PR18MB029692D26E757F38AF7A44F299260@BY2PR18MB0296.namprd18.prod.outlook.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [31.53.131.180]
x-source-routing-agent: Processed
Content-Type: text/plain; charset="us-ascii"
Content-ID: <7265E136F3EB62498030867DE348EE69@exch080.serverpod.net>
Content-Transfer-Encoding: quoted-printable
archived-at: Thu, 16 Mar 2017 12:04:21 -0000

some of this looks network related. Ubuntu often makes a mess of the loopba=
ck address of localhost: find the localhost entry in /etc/hosts and change =
it to 127.0.0.1 and see if that helps

otherwise, the hdfs-dev list is the one to really ping about HDFS issues, w=
hich all of these appear to be. Mostly related to XAttrs, interestingly eno=
ugh


Pick one isolated test, like testWebHDFSXAttr#FSXAttrBaseTest.testListXAttr=
 ; run it with the log4j.properties in hdfs src/test/resources set to log o=
rg.apache.hadoop at debug and see what is going on underneath. That's gener=
ally how I start debugging test failures, before switching to the IDE and s=
tepping through.

Good luck

Steve

PS if it makes you feel any better, staring a test run failures is a core s=
kill of all Hadoop developers


> On 16 Mar 2017, at 10:33, Simon Scott <Simon.Scott@viavisolutions.com> wr=
ote:
>=20
> Hi
>=20
> Please excuse me. I am new to building Hadoop. I am attempting to build a=
nd run tests for the trunk and encountering a series of test failures which=
 for sure indicates that I am doing something wrong. I have scoured the wik=
i and can't see what I'm missing. Can anybody offer any advice?
>=20
> I am running the tests in Ubuntu 14.04 hosted in VirtualBox on Windows 8.=
1 host.
>=20
> Using this command: "mvn clean test"
>=20
> List of failed tests below.
>=20
> Many thanks
> Simon
>=20
> Failed tests:
>  TestHDFSFileSystemContract>FileSystemContractBaseTest.testRenameDirector=
yAsExistingDirectory:536 Renamed nested file1 exists
>  TestTrashWithEncryptionZones.testDeleteEZWithMultipleUsers:164 Non-admin=
 could delete an encryption zone with multiple users : /zones/zone1 expecte=
d:<1> but was:<0>
>  TestDataNodeVolumeFailure.testFailedVolumeBeingRemovedFromDataNode:247 e=
xpected:<1> but was:<2>
>  TestFileContextXAttr>FSXAttrBaseTest.testUnreadableBySuperuserXAttr:1142=
->FSXAttrBaseTest.access$000:67->FSXAttrBaseTest.doTestUnreadableBySuperuse=
rXAttr:1158->FSXAttrBaseTest.doTUBSXAInt:1213->FSXAttrBaseTest.verifyFileAc=
cess:1241 open failed but expected it to succeed
>  TestFileContextXAttr>FSXAttrBaseTest.testRawXAttrs:1024 setXAttr should =
have thrown
>  TestFileContextXAttr>FSXAttrBaseTest.testListXAttrs:820 expected IOExcep=
tion
>  TestFileContextXAttr>FSXAttrBaseTest.testGetXAttrs:458 expected IOExcept=
ion
>  TestNameNodeXAttr>FSXAttrBaseTest.testUnreadableBySuperuserXAttr:1142->F=
SXAttrBaseTest.access$000:67->FSXAttrBaseTest.doTestUnreadableBySuperuserXA=
ttr:1158->FSXAttrBaseTest.doTUBSXAInt:1213->FSXAttrBaseTest.verifyFileAcces=
s:1241 open failed but expected it to succeed
>  TestNameNodeXAttr>FSXAttrBaseTest.testRawXAttrs:1024 setXAttr should hav=
e thrown
>  TestNameNodeXAttr>FSXAttrBaseTest.testListXAttrs:820 expected IOExceptio=
n
>  TestNameNodeXAttr>FSXAttrBaseTest.testGetXAttrs:458 expected IOException
>  TestNamenodeCapacityReport.testXceiverCount:198->testXceiverCountInterna=
l:344 Test resulted in an unexpected exit
>  TestEncryptionZonesWithKMS>TestEncryptionZones.testGetEZAsNonSuperUser:5=
81 expected AccessControlException
>  TestEncryptionZonesWithKMS>TestEncryptionZones.testBasicOperations:441 c=
reateEncryptionZone is superuser-only operation
>  TestReservedRawPaths.testAdminAccessOnly:262 access to /.reserved/raw is=
 superuser-only operation
>  TestDFSShell.testSetXAttrPermission:2992 Returned should be 1 expected:<=
1> but was:<0>
>  TestWebHDFSXAttr>FSXAttrBaseTest.testUnreadableBySuperuserXAttr:1142->FS=
XAttrBaseTest.access$000:67->FSXAttrBaseTest.doTestUnreadableBySuperuserXAt=
tr:1158->FSXAttrBaseTest.doTUBSXAInt:1213->FSXAttrBaseTest.verifyFileAccess=
:1241 open failed but expected it to succeed
>  TestWebHDFSXAttr>FSXAttrBaseTest.testRawXAttrs:1024 setXAttr should have=
 thrown
>  TestWebHDFSXAttr>FSXAttrBaseTest.testListXAttrs:820 expected IOException
>  TestWebHDFSXAttr>FSXAttrBaseTest.testGetXAttrs:458 expected IOException
>  TestWebHdfsFileSystemContract>FileSystemContractBaseTest.testRenameDirec=
toryAsExistingDirectory:536 Renamed nested file1 exists
>  TestEncryptionZones.testGetEZAsNonSuperUser:581 expected AccessControlEx=
ception
>  TestEncryptionZones.testBasicOperations:441 createEncryptionZone is supe=
ruser-only operation
>=20
> Tests in error:
>  TestRBWBlockInvalidation.testBlockInvalidationWhenRBWReplicaMissedInDN:1=
22 >  ...
>  TestDataNodeVolumeFailure.tearDown:142->Object.wait:-2 >  test timed out=
 after...
>  TestDataNodeVolumeFailureReporting.tearDown:109->Object.wait:-2 >  test =
timed ...
>  TestDataNodeVolumeFailureReporting.tearDown:109 >  test timed out after =
120000...
>  TestDataNodeVolumeFailureReporting.tearDown:109 >  test timed out after =
120000...
>  TestDataNodeVolumeFailureReporting.tearDown:109->Object.wait:-2 >  test =
timed ...
>  TestFileContextXAttr>FSXAttrBaseTest.testRemoveXAttrPermissions:656 > Re=
mote N...
>  TestNameNodeMetadataConsistency.testGenerationStampInFuture:109->waitFor=
NumBytes:161 > Timeout
>  TestNameNodeXAttr>FSXAttrBaseTest.testRemoveXAttrPermissions:656 > Remot=
e No m...
>  TestFSImage.testCompression:96->setCompressCodec:102->testPersistHelper:=
108 > IO
>  TestDiskBalancerCommand.testRunMultipleCommandsUnderOneSetup > Remote Fi=
le /sy...
>  TestDiskBalancer.testBalanceDataBetweenMultiplePairsOfVolumes:187 > Ille=
galArgument
>  TestDiskBalancerRPC.testMoveBlockAcrossVolume > Remote File /tmp.txt cou=
ld onl...
>  TestDFSRSDefault10x4StripedOutputStream>TestDFSStripedOutputStream.setup=
:91 > DiskError
>  TestDFSRSDefault10x4StripedOutputStream>TestDFSStripedOutputStream.testF=
ileSmallerThanOneStripe2:132->TestDFSStripedOutputStream.testOneFile:187 > =
Timeout
>  TestDFSRSDefault10x4StripedOutputStream>TestDFSStripedOutputStream.testF=
ileMoreThanABlockGroup3:176->TestDFSStripedOutputStream.testOneFile:186 > I=
O
>  TestParallelRead.setupCluster:37->TestParallelReadUtil.setupCluster:71 >=
 IO Ca...


>  TestParallelRead.teardownCluster:42->TestParallelReadUtil.teardownCluste=
r:394 > NullPointer

that's a clear bug in the teardown logic; I'd run that standalone then file=
 a JIRA on it...it's not checking for a field being set before invoking. Ro=
ot cause is probably a failure in test setup tho


>  TestSaslDataTransfer.testNoSaslAndSecurePortsIgnored:175->startCluster:2=
07 > FileNotFound
>  TestDatanodeLayoutUpgrade.testUpgradeToIdBasedLayout:52 > IO Timed out w=
aiting...
>  TestDFSMkdirs.testMkdir:94 > DiskError Too many failed volumes - current=
 valid...
>  TestWebHDFSXAttr>FSXAttrBaseTest.testRemoveXAttrPermissions:656 > IO No =
matchi...
>=20
> Tests run: 4997, Failures: 23, Errors: 22, Skipped: 150


---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org