hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10014) Symlink resolution is fundamentally broken for multi-layered filesystems
Date Fri, 04 Oct 2013 16:35:43 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786302#comment-13786302

Daryn Sharp commented on HADOOP-10014:

I think it's going to be dicey and complicated to pass context, especially w/o incompatible
fs subclass changes, but it'd be interesting to see a POC.

Basically the path resolution needs to occur from the top down.  I think it's ok for individual
filesystem to provide primitives like a getFileLinkStatus, getLinkTarget, resolveLink, listLinkStatus,
etc.  But transparent resolution for the existing API methods needs to occur from the top-down
which is why it may simpler to apply a "SymlinkFileSystem" derived from {{FilterFileSystem}}
atop an arbitrary fs.  At that rate, maybe the existing APIs should return symlinks and the
symlink fs should handle resolving them...  FileSystem.get would then return a wrapped fs
if symlinks are enabled.

> Symlink resolution is fundamentally broken for multi-layered filesystems
> ------------------------------------------------------------------------
>                 Key: HADOOP-10014
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10014
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Daryn Sharp
>            Priority: Critical
> Symlink resolution is performed on a per-filesystem basis.  In a multi-layered filesystem,
the symlinks need to be resolved relative to the highest level filesystem in the stack.  Otherwise,
fs implementations like viewfs and chroot fs behave incorrectly.  Absolute symlinks may violate
the base of the chroot.  Links that should have crossed viewfs mount points are again incorrectly
resolved relative to the base filesystem.
> Symlink resolution has occur above the level of any individual fs to allow a multi-layered
fs stack to work correctly, such as via a symlink-aware {{FilteredFileSystem}} that wraps
any arbitrary fs to ensure links are resolved from the top-down of the stack.

This message was sent by Atlassian JIRA

View raw message