hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-245) Create symbolic links in HDFS
Date Fri, 25 Sep 2009 15:58:17 GMT

    [ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12759595#action_12759595

Doug Cutting commented on HDFS-245:

> The FileContext API is not yet clearly baked as far as I can tell.

It's committed.  It needs to get used.  Adding the symlink feature there will get both used
and tested without altering existing codepaths.

> I'm fine with only doing it in file context... if we wait to checkin symlinks until FileSystem
is fully deprecated.

I don't follow your logic here, Eric.  Doesn't it make sense to first add a new API in a trial
manner?  Pioneers can work out issues with symbolic links before others are forced to move
to the new API.

Symbolic links can be added to FileContext such a way that the only changes to FileSystem
would be to add a method, getLinkTarget(), that no existing code calls.  So the changes to
Common are very low risk this way.

Separately we can then have a patch that adds symbolic link support to HDFS.  The primary
change to existing codepaths here would be changing places where FileNotFoundException would
be thrown to instead throw UnresolvedLinkException when the path to the file contains a symbolic
link.  The larger risk/complication in HDFS is that links would be added to FileStatus, and
hence the namenode log format would need to be extended and we'd need a downgrade story. 
So this patch would have more risk.  But it can be considered separately.

Additionally we can have patches that add support for local FS, S3, etc.

> Create symbolic links in HDFS
> -----------------------------
>                 Key: HDFS-245
>                 URL: https://issues.apache.org/jira/browse/HDFS-245
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>         Attachments: 4044_20081030spi.java, designdocv1.txt, HADOOP-4044-strawman.patch,
symlink-0.20.0.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch,
symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch,
symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symLink4.patch, symLink5.patch,
symLink6.patch, symLink8.patch, symLink9.patch
> HDFS should support symbolic links. A symbolic link is a special type of file that contains
a reference to another file or directory in the form of an absolute or relative path and that
affects pathname resolution. Programs which read or write to files named by a symbolic link
will behave as if operating directly on the target file. However, archiving utilities can
handle symbolic links specially and manipulate them directly.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message