hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjay Radia (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8589) ViewFs tests fail when tests and home dirs are nested
Date Fri, 12 Oct 2012 02:35:02 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13474740#comment-13474740

Sanjay Radia commented on HADOOP-8589:

My patch does the following (well commented in the patch it self)
# sets the mount points = 
+ * We set a viewFileSystems with 3 mount points: 
+ * 1) /<firstComponent>" of testdir  pointing to same in  target fs
+ * 2)   /<firstComponent>" of home  pointing to same in  target fs 
+ * 3)  /<firstComponent>" of wd  pointing to same in  target fs
+ * (note in many cases the link may be the same - viewFileSytem handles this)
# sets viewfs's wd  since otherwise if you runs the tests in eclipse it create the testdirs
in your home dir.
# teardown ensures that junk is not left in your home dir (due to previous wd fix).
# the above changes are made for BOTH viewfs and viewFileSystem tests
# a couple of tests now calls the common setup routines
# the chroot test fails on mac - fixed that

I tried  doing some of the above in separate patches but then some tests were failing in either
mac or linux.

My patch does not fix trash issue. I believe this should be separate patch (but if you prefer
i can incorporate it here).
However the TestLFS should be moved back to where it was as I have indicated in a previous
in your patch why did you move TestLFS from TestViewFsTrash to ViewFileSystemTestSetup?
While ViewFileSystemTestSetup is used in many tests, TestLFS is used in only in ViewFileSystemTestSetup.

If you prefer we can let your patch through (but move the TestLFS back to TestViewFsTrash
since it is used only there).
I can then do the cleanup I want to do in a separate jira. 
> ViewFs tests fail when tests and home dirs are nested
> -----------------------------------------------------
>                 Key: HADOOP-8589
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8589
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs, test
>    Affects Versions: 0.23.1, 2.0.0-alpha
>            Reporter: Andrey Klochkov
>            Assignee: Andrey Klochkov
>         Attachments: HADOOP-8589.patch, HADOOP-8589.patch, hadoop-8589-sanjay.patch,
> TestFSMainOperationsLocalFileSystem fails in case when the test root directory is under
the user's home directory, and the user's home dir is deeper than 2 levels from /. This happens
with the default 1-node installation of Jenkins. 
> This is the failure log:
> {code}
> org.apache.hadoop.fs.FileAlreadyExistsException: Path /var already exists as dir; cannot
create link here
> 	at org.apache.hadoop.fs.viewfs.InodeTree.createLink(InodeTree.java:244)
> 	at org.apache.hadoop.fs.viewfs.InodeTree.<init>(InodeTree.java:334)
> 	at org.apache.hadoop.fs.viewfs.ViewFileSystem$1.<init>(ViewFileSystem.java:167)
> 	at org.apache.hadoop.fs.viewfs.ViewFileSystem.initialize(ViewFileSystem.java:167)
> 	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2094)
> 	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:79)
> 	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2128)
> 	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2110)
> 	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:290)
> 	at org.apache.hadoop.fs.viewfs.ViewFileSystemTestSetup.setupForViewFileSystem(ViewFileSystemTestSetup.java:76)
> 	at org.apache.hadoop.fs.viewfs.TestFSMainOperationsLocalFileSystem.setUp(TestFSMainOperationsLocalFileSystem.java:40)
> ...
> Standard Output
> 2012-07-11 22:07:20,239 INFO  mortbay.log (Slf4jLog.java:info(67)) - Home dir base /var/lib
> {code}
> The reason for the failure is that the code tries to mount links for both "/var" and
"/var/lib", and it fails for the 2nd one as the "/var" is mounted already.
> The fix was provided in HADOOP-8036 but later it was reverted in HADOOP-8129.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message