hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9984) FileSystem#globStatus and FileSystem#listStatus should resolve symlinks by default
Date Thu, 03 Oct 2013 05:26:46 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13784835#comment-13784835

Suresh Srinivas commented on HADOOP-9984:

[~cmccabe], some comments:
bq. Making listLinkStatus an abstract function is actually better for these out-of-tree implementors
Adding abstract function is one way. The other way is by adding default implementation that
throw unsupported exception. The second approach has been taken for most of the functionality
added later into the file system including createSymlink. Symbolic links is a capability not
all the file systems may support. We should follow the same approach for this functionality
as well.

bq. I can say firsthand that no matter what option we choose, the ceph hadoop plugin will
need to be updated (I worked on that at one point).
Only if you want to add symlink functionality, right?

There are two kinds of incompatibilities we are talking about:
# FileSystem API as used by the applications. I agree with you that not breaking compatibility
here that requires application code change is the highest priority.
# FileSystem implementations. While breaking compatibility here may be fine, I believe we
can avoid it.

bq. Should we have a webex tomorrow to resolve this? This issue is a blocker and it would
be nice to resolve it soon
I agree that this needs to be quickly resolved. Lets have a webex. I also want to understand
why HADOOP-9972 has to be incompatible.

> FileSystem#globStatus and FileSystem#listStatus should resolve symlinks by default
> ----------------------------------------------------------------------------------
>                 Key: HADOOP-9984
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9984
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 2.1.0-beta
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>            Priority: Blocker
>         Attachments: HADOOP-9984.001.patch, HADOOP-9984.003.patch, HADOOP-9984.005.patch,
HADOOP-9984.007.patch, HADOOP-9984.009.patch, HADOOP-9984.010.patch, HADOOP-9984.011.patch,
HADOOP-9984.012.patch, HADOOP-9984.013.patch, HADOOP-9984.014.patch
> During the process of adding symlink support to FileSystem, we realized that many existing
HDFS clients would be broken by listStatus and globStatus returning symlinks.  One example
is applications that assume that !FileStatus#isFile implies that the inode is a directory.
 As we discussed in HADOOP-9972 and HADOOP-9912, we should default these APIs to returning
resolved paths.

This message was sent by Atlassian JIRA

View raw message