Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 06A1511AA7 for ; Wed, 4 Jun 2014 16:21:59 +0000 (UTC) Received: (qmail 98114 invoked by uid 500); 4 Jun 2014 16:21:54 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 97978 invoked by uid 500); 4 Jun 2014 16:21:54 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 97971 invoked by uid 99); 4 Jun 2014 16:21:54 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 Jun 2014 16:21:54 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of yuzhihong@gmail.com designates 209.85.213.46 as permitted sender) Received: from [209.85.213.46] (HELO mail-yh0-f46.google.com) (209.85.213.46) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 Jun 2014 16:21:49 +0000 Received: by mail-yh0-f46.google.com with SMTP id 29so6618347yhl.19 for ; Wed, 04 Jun 2014 09:21:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=Se9kYOWTmQEboedBL5Jen7EmRATwsJi6JC6o7nsQhdc=; b=f04Z2cWiB4g9Sd/vKJeSnm6zDlN5KBAuTylBaA3lRpTbdDLLvpvd1fuJ0A+utuekDr 3/zvRTLbDzXn7kPa+RfPYtTVqA45dyEtbOq+/L+AZVRA3ONndJgi4kdtngT4N1eMajxb Ibq2FOMEM5P5pxp/MjEy/rHfoGbBZcdIEIJduRovmCgmdRTJw7dV33IK49osqQAr/n21 03apTS6oP8Lfiz2TSe76uSZGUHCDPfP6tPDSDNbGnZbpA0GKaL+S8Jg5NcxDZBcrgNOR 4wqtpNZ9ydd9f4loAUuXPpFuQ7dUoEAiDXVUDlzS1hTHdrOf74lxLRB5z5H1CTgpGsME heSw== MIME-Version: 1.0 X-Received: by 10.236.141.228 with SMTP id g64mr69625117yhj.73.1401898888291; Wed, 04 Jun 2014 09:21:28 -0700 (PDT) Received: by 10.170.37.144 with HTTP; Wed, 4 Jun 2014 09:21:28 -0700 (PDT) In-Reply-To: References: Date: Wed, 4 Jun 2014 09:21:28 -0700 Message-ID: Subject: Re: (Very) newbie questions From: Ted Yu To: "common-user@hadoop.apache.org" Content-Type: multipart/alternative; boundary=20cf303a2f3f3f69b204fb05055e X-Virus-Checked: Checked by ClamAV on apache.org --20cf303a2f3f3f69b204fb05055e Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable To run a single test, you can use: mvn test -Dtest=3DTestSymlinkLocalFSFileContext Adding the assertion message shown below would give more information: Index: hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/Syml= inkBaseTest.java =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/Syml= inkBaseTest.java (revision 1599851) +++ hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/Syml= inkBaseTest.java (working copy) @@ -374,7 +374,7 @@ assertEquals(fileSize, wrapper.getFileStatus(linkAbs).getLen()); // Check getFileLinkStatus - assertTrue(wrapper.isSymlink(linkAbs)); + assertTrue(linkAbs + ": " + wrapper.getFileLinkStatus(linkAbs), wrapper.isSymlink(linkAbs)); assertFalse(wrapper.getFileLinkStatus(linkAbs).isDirectory()); // Check getSymlink always returns a qualified target, except On Wed, Jun 4, 2014 at 9:08 AM, Christian Convey wrote: > I'm going to try looking into it today, although I'm not sure how quickly > I'll figure it out since I'm new to Maven. FYI, I'm getting this using > Linux Mint 17 32-bit MATE edition, inside a VirtualBox VM, on OS X. > > > On Wed, Jun 4, 2014 at 11:59 AM, Ted Yu wrote: > >> Thanks for the pointer, Tsuyoshi. >> >> I ran TestSymlinkLocalFSFileContext on Mac and Linux - the test passed. >> >> I think wrapper.getFileLinkStatus() should be added to assertion message >> when assertTrue(wrapper.isSymlink(linkAbs)) fails. This would give us >> more information. >> >> Cheers >> >> >> On Tue, Jun 3, 2014 at 5:53 PM, Tsuyoshi OZAWA >> wrote: >> >>> Hi Ted and Christian, >>> >>> In fact, the problem is filed as HADOOP-10510. >>> https://issues.apache.org/jira/browse/HADOOP-10510 >>> This problem is also reproduced on my local, but not on the other >>> environment. Maybe it's environment-dependent problem. However, I >>> cannot understand the condition this problem occurs correctly. >>> >>> Thanks, >>> - Tsuyoshi >>> >>> >>> On Tue, Jun 3, 2014 at 5:49 PM, Ted Yu wrote: >>> > TestSymlinkLocalFS is an abstract class. >>> > >>> > The actual test is in TestSymlinkLocalFSFileContext / >>> > TestSymlinkLocalFSFileSystem. >>> > Do they pass on your computer ? >>> > >>> > >>> > >>> > On Tue, Jun 3, 2014 at 5:21 PM, Christian Convey >>> > wrote: >>> >> >>> >> >>> >> On Tue, Jun 3, 2014 at 8:19 PM, Christian Convey >>> >> wrote: >>> >>> >>> >>> >>> >>> >>> >>> On Tue, Jun 3, 2014 at 6:37 PM, Ted Yu wrote: >>> >>>> >>> >>>> Do you mind listing the unit tests that failed on your computer ? >>> >>>> >>> >>>> Cheers >>> >>>> >>> >>>> >>> >>>> On Tue, Jun 3, 2014 at 3:31 PM, Christian Convey >>> >>>> wrote: >>> >>>>> >>> >>>>> I'm completely new to Hadoop, and I'm trying to build it for the >>> first >>> >>>>> time. I cloned the Git repository and I'm building the tag versi= on >>> >>>>> release-2.4.0. >>> >>>>> >>> >>>>> I get a clean "mvn compile -Pnative", but I'm getting a few unit >>> tests >>> >>>>> failing when I run "mvn test". >>> >>>>> >>> >>>>> Does anyone know to what extent Hadoop (Common) 's unit tests >>> failing >>> >>>>> is a serious issue? I.e., can I have a healthy Hadoop build, >>> while still >>> >>>>> having had a few unit tests fail? >>> >>>> >>> >>>> >>> >>> >>> >>> The one that jumped out me is >>> >>> "org.apache.hadoop.fs.TestSymlinkLocalFSFileSystem". There may be >>> others, >>> >>> but I haven't noticed them. >>> >> >>> >> >>> >> Actually, I believe this is the pertinent output: >>> >> >>> >> >>> >> Tests in error: >>> >> >>> TestSymlinkLocalFSFileContext>TestSymlinkLocalFS.testDanglingLink:163 = =C2=BB >>> >> IO Pat... >>> >> >>> >> >>> TestSymlinkLocalFSFileContext>TestSymlinkLocalFS.testGetLinkStatusPartQ= ualTarget:201 >>> >> =C2=BB IO >>> >> >>> >> >>> TestSymlinkLocalFSFileContext>SymlinkBaseTest.testCreateLinkToDotDotPre= fix:822 >>> >> =C2=BB IO >>> >> >>> TestSymlinkLocalFSFileSystem>TestSymlinkLocalFS.testDanglingLink:163 = =C2=BB >>> >> IO Path... >>> >> >>> >> >>> TestSymlinkLocalFSFileSystem>TestSymlinkLocalFS.testGetLinkStatusPartQu= alTarget:201 >>> >> =C2=BB IO >>> >> >>> >> >>> TestSymlinkLocalFSFileSystem>SymlinkBaseTest.testCreateLinkToDotDotPref= ix:822 >>> >> =C2=BB IO >>> >> >>> >> Tests run: 2285, Failures: 24, Errors: 6, Skipped: 41 >>> >> >>> >> >>> > >>> >>> >>> >>> -- >>> - Tsuyoshi >>> >> >> > --20cf303a2f3f3f69b204fb05055e Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
To run a single test, you can use:

mvn test -Dtest=3DTestSymlinkLocalFSFileContext

Adding the assertion message shown below would give more information:
Index: hadoop-common-project/hadoop-common/src/test/java/org/= apache/hadoop/fs/SymlinkBaseTest.java
=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D
--- hadoop-common-project/hadoop-common/= src/test/java/org/apache/hadoop/fs/SymlinkBaseTest.java (revision 1599851)
+++ hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoo= p/fs/SymlinkBaseTest.java (working copy)
@@ -374,7 +374,7 @@
=C2=A0 =C2=A0 =C2= =A0assertEquals(fileSize, wrapper.getFileStatus(linkAbs).getLen());

=C2=A0 =C2=A0 =C2=A0// Check getFileLinkStatus
- =C2=A0 =C2=A0assertTrue(wrapper.isSymlink(linkAbs));
+ =C2=A0= =C2=A0assertTrue(linkAbs + ": " + wrapper.getFileLinkStatus(link= Abs), wrapper.isSymlink(linkAbs));
=C2=A0 =C2=A0 =C2=A0assertFalse(wrapper.getFileLinkStatus(linkAbs).isD= irectory());

=C2=A0 =C2=A0 =C2=A0// Check getSymli= nk always returns a qualified target, except


On Wed, Jun 4, 2014 at 9:08 AM, Christian Convey= <christian.convey@gmail.com> wrote:
I'm going to try looking into it today, although I'= ;m not sure how quickly I'll figure it out since I'm new to Maven. = =C2=A0FYI, I'm getting this using Linux Mint 17 32-bit MATE edition, in= side a VirtualBox VM, on =C2=A0OS X.


On Wed, Jun 4= , 2014 at 11:59 AM, Ted Yu <yuzhihong@gmail.com> wrote:
Thanks for the pointer,=C2=A0Tsuyoshi.

I ran=C2=A0TestSymlinkLocalFS= FileContext on Mac and Linux - the test passed.

I think wrapper.getFileLinkStatus() should b= e added to assertion message when assertTrue(wrapper.isSymlink= (linkAbs)) fails. This would give us more information.

Cheers


On= Tue, Jun 3, 2014 at 5:53 PM, Tsuyoshi OZAWA <ozawa.tsuyoshi@gmail= .com> wrote:
Hi Ted and Christian,

In fact, the problem is filed as HADOOP-10510.
https://issues.apache.org/jira/browse/HADOOP-10510
This problem is also reproduced on my local, but not on the other
environment. Maybe it's environment-dependent problem. However, I
cannot understand the condition this problem occurs correctly.

Thanks,
- Tsuyoshi


On Tue, Jun 3, 2014 at 5:49 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> TestSymlinkLocalFS is an abstract class.
>
> The actual test is in TestSymlinkLocalFSFileContext /
> TestSymlinkLocalFSFileSystem.
> Do they pass on your computer ?
>
>
>
> On Tue, Jun 3, 2014 at 5:21 PM, Christian Convey
> <ch= ristian.convey@gmail.com> wrote:
>>
>>
>> On Tue, Jun 3, 2014 at 8:19 PM, Christian Convey
>> <christian.convey@gmail.com> wrote:
>>>
>>>
>>>
>>> On Tue, Jun 3, 2014 at 6:37 PM, Ted Yu <yuzhihong@gmail.com> wrote: >>>>
>>>> Do you mind listing the unit tests that failed on your com= puter ?
>>>>
>>>> Cheers
>>>>
>>>>
>>>> On Tue, Jun 3, 2014 at 3:31 PM, Christian Convey
>>>> <christian.convey@gmail.com> wrote:
>>>>>
>>>>> I'm completely new to Hadoop, and I'm trying t= o build it for the first
>>>>> time. =C2=A0I cloned the Git repository and I'm bu= ilding the tag version
>>>>> release-2.4.0.
>>>>>
>>>>> I get a clean "mvn compile -Pnative", but I&= #39;m getting a few unit tests
>>>>> failing when I run "mvn test".
>>>>>
>>>>> Does anyone know to what extent Hadoop (Common) 's= unit tests failing
>>>>> is a serious issue? =C2=A0I.e., can I have a healthy H= adoop build, while still
>>>>> having had a few unit tests fail?
>>>>
>>>>
>>>
>>> The one that jumped out me is
>>> "org.apache.hadoop.fs.TestSymlinkLocalFSFileSystem".= =C2=A0There may be others,
>>> but I haven't noticed them.
>>
>>
>> Actually, I believe this is the pertinent output:
>>
>>
>> Tests in error:
>> =C2=A0 TestSymlinkLocalFSFileContext>TestSymlinkLocalFS.testDan= glingLink:163 =C2=BB
>> IO Pat...
>>
>> TestSymlinkLocalFSFileContext>TestSymlinkLocalFS.testGetLinkSta= tusPartQualTarget:201
>> =C2=BB IO
>>
>> TestSymlinkLocalFSFileContext>SymlinkBaseTest.testCreateLinkToD= otDotPrefix:822
>> =C2=BB IO
>> =C2=A0 TestSymlinkLocalFSFileSystem>TestSymlinkLocalFS.testDang= lingLink:163 =C2=BB
>> IO Path...
>>
>> TestSymlinkLocalFSFileSystem>TestSymlinkLocalFS.testGetLinkStat= usPartQualTarget:201
>> =C2=BB IO
>>
>> TestSymlinkLocalFSFileSystem>SymlinkBaseTest.testCreateLinkToDo= tDotPrefix:822
>> =C2=BB IO
>>
>> Tests run: 2285, Failures: 24, Errors: 6, Skipped: 41
>>
>>
>



--
- Tsuyoshi



--20cf303a2f3f3f69b204fb05055e--