hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjay Radia (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4044) Create symbolic links in HDFS
Date Tue, 14 Oct 2008 21:47:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639603#action_12639603

Sanjay Radia commented on HADOOP-4044:

_Doug Says: >Internal symlinks are supported automatically by local FS, at least on unix.
And support for remote symlinks on the local FS might be very cool. Unix will gladly let me
create symbolics link to anHDFS URI (Try 'ln -s hdfs://foo/bar foo; ls -l foo'). These cannot
be resolved by a unix shell, but they could by 'bin/hadoop fs -ls', by MapReduce programs
etc. One could, e.g., keep in one's home directory on unix links to the HDFS directories that
one commonly uses .._

 It would be very cool to support them. (I guess unix thinks the : is simply part of the name.)
BTW  if someone happens to create a dir called "hdfs:" and valid path foo/bar under it then
the local file system will follow it and not return an error. Is this a concern?

When trying to go through the remote mount symlink,  the local file system  reports that the
file does not exist. Hence, on a "file does not exist" error, the client side lib will have
to check if there are  symlinks in the path. The only snag is that "file not found error"s
will take a little longer to report as the 
library ties to check if there is a symlink in the path. Is this an issue? Most apps consider
"file not found" to be an error and hence the speed of reporting that is not a concern; however,
there are  some apps that create files if they are missing and their performance will suffer
a little. 

> Create symbolic links in HDFS
> -----------------------------
>                 Key: HADOOP-4044
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4044
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>         Attachments: HADOOP-4044-strawman.patch, symLink1.patch, symLink1.patch, symLink4.patch,
symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch
> HDFS should support symbolic links. A symbolic link is a special type of file that contains
a reference to another file or directory in the form of an absolute or relative path and that
affects pathname resolution. Programs which read or write to files named by a symbolic link
will behave as if operating directly on the target file. However, archiving utilities can
handle symbolic links specially and manipulate them directly.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message