hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9984) FileSystem#globStatus and FileSystem#listStatus should resolve symlinks by default
Date Fri, 27 Sep 2013 21:14:05 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13780411#comment-13780411
] 

Colin Patrick McCabe commented on HADOOP-9984:
----------------------------------------------

bq. Both listStatus and listLinkStatus must return file stats with the exact same paths. Those
paths must be constructed from the "unresolved" paths. The only difference is whether the
stat is for the symlink itself, or the stat of the resolved path.

Here's an illustration.  If you have inodes like this:
{code}
a symlink: /a -> /b
a directory: /b
a file: /b/c
{code}

You get:
{code}
getFileStatus("/a/b/c") returns FileStatus(path="/b/c", type:file)
getFileLinkStatus("/a/b/c") returns FileStatus(path="/b/c", type:file)
getFileStatus("/a") returns FileStatus(path="/b", type:dir)
getFileLinkStatus("/a") returns FileStatus(path="/a", type:symlink)
{code}

{{getFileStatus}} and {{listStatus}} have *always* resolved intermediate path components.
 What we're discussing here is whether {{listStatus}} should resolve the final path component,
like {{getFileStatus}} does.

Basically, {{listLinkStatus}} is *exactly* like {{getFileLinkStatus}}, but for directory listing.
                
> FileSystem#globStatus and FileSystem#listStatus should resolve symlinks by default
> ----------------------------------------------------------------------------------
>
>                 Key: HADOOP-9984
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9984
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 2.1.0-beta
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>            Priority: Blocker
>         Attachments: HADOOP-9984.001.patch, HADOOP-9984.003.patch, HADOOP-9984.005.patch
>
>
> During the process of adding symlink support to FileSystem, we realized that many existing
HDFS clients would be broken by listStatus and globStatus returning symlinks.  One example
is applications that assume that !FileStatus#isFile implies that the inode is a directory.
 As we discussed in HADOOP-9972 and HADOOP-9912, we should default these APIs to returning
resolved paths.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message