hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9984) FileSystem#globStatus and FileSystem#listStatus should resolve symlinks by default
Date Thu, 03 Oct 2013 16:28:43 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13785319#comment-13785319

Chris Nauroth commented on HADOOP-9984:

Another one is {{NativeAzureFileSystem}}.  This has been contributed in HADOOP-9629, but not
reviewed and committed, so for now it lives out-of-tree.  Conceptually, it's most similar
to the Swift implementation (adapter over a blobstore, and no symlink support).  It's trivial
to fix it after this patch, though perhaps a bit confusing that a {{FileSystem}} with no symlink
support must implement methods with "link" in the name.

Another potential impact is subclasses that are implementation details or testing infrastructure
inside Hadoop ecosystem projects.  (In fact, this was my primary concern when I was talking
about "downstream projects".)  An example is Hive's {{ProxyFileSystem}}.  That one extends
{{FilterFileSystem}}, so it's going to be fine, but if something similar extends {{FileSystem}},
then it will break.  A full recompile of the whole ecosystem would tell us for sure.

> FileSystem#globStatus and FileSystem#listStatus should resolve symlinks by default
> ----------------------------------------------------------------------------------
>                 Key: HADOOP-9984
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9984
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 2.1.0-beta
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>            Priority: Blocker
>         Attachments: HADOOP-9984.001.patch, HADOOP-9984.003.patch, HADOOP-9984.005.patch,
HADOOP-9984.007.patch, HADOOP-9984.009.patch, HADOOP-9984.010.patch, HADOOP-9984.011.patch,
HADOOP-9984.012.patch, HADOOP-9984.013.patch, HADOOP-9984.014.patch
> During the process of adding symlink support to FileSystem, we realized that many existing
HDFS clients would be broken by listStatus and globStatus returning symlinks.  One example
is applications that assume that !FileStatus#isFile implies that the inode is a directory.
 As we discussed in HADOOP-9972 and HADOOP-9912, we should default these APIs to returning
resolved paths.

This message was sent by Atlassian JIRA

View raw message