hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5293) Symlink resolution requires unnecessary RPCs
Date Fri, 04 Oct 2013 00:15:42 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13785706#comment-13785706
] 

Colin Patrick McCabe commented on HDFS-5293:
--------------------------------------------

If we're going to do this, why not just make all NN operations resolve symlinks as far as
they can?  That would remove all the performance concerns about returning unresolved paths,
at least in the context of non-cross-FS symlinks.

We already have many filesystems that do symlink resolution internally, such as LocalFileSystem,
Ceph, etc. etc.  If HDFS's namenode did symlink resolution for all RPCs, we could return unresolved
paths everywhere and be happy.

> Symlink resolution requires unnecessary RPCs
> --------------------------------------------
>
>                 Key: HDFS-5293
>                 URL: https://issues.apache.org/jira/browse/HDFS-5293
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Daryn Sharp
>            Priority: Critical
>
> When the NN encounters a symlink, it throws an {{UnresolvedLinkException}}.  This exception
contains only the path that is a symlink.  The client issues another RPC to obtain the link
target, followed by another RPC with the link target + remainder of the original path.
> {{UnresolvedLinkException}} should be returning both the link and the target to avoid
a costly and unnecessary intermediate RPC to obtain the link target.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message