hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HDFS-245) Create symbolic links in HDFS
Date Wed, 03 Feb 2010 06:55:42 GMT

     [ https://issues.apache.org/jira/browse/HDFS-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Eli Collins updated HDFS-245:

    Attachment: symlink36-hdfs.patch

Hey Hairong, thanks for taking a look!

bq. 1. INodeSymlink extends INodeFile in your patch. Should it extend INode?

The current code extends INodeFile because logically a symlink is a file (that just contains
a path), and it makes the change smaller (by avoiding overriding spaceConsumedInTree, collectSubtreeBlocksAndClear,
computeContentSummary, and allowing all the code that handles INodeFile to cover files and
symlinks). I like your suggestion though; explicitly differentiating between file and symlink
inodes seems less error prone, and there's a minor win in that the code no longer has to interpret
and serialize the INodeFile fields that are unused for symlinks. I made INodeSymlink extend
INode and modified FSImage#loadFSImage and saveINode2Image, and ImageLoaderCurrent accordingly.
Leveraged the existing method of using a negative block size value to indicate the inode type.
I also had to add a couple additional checks INode#isLink in places where we were leveraging
the fact that symlinks were files.

2. Should FSDirectory#mkdirs, rootDir.getExistingPathINodes(components, inodes, true) be rootDir.getExistingPathINodes(components,
inodes, false)? Mkdirs does not need to resolve the symlink of the last component.

Yup, there's no need for mkdir to try resolve the final link. I also added throws UnresolvedLinkException
which was missing from the javadoc.

3. Your document does not discuss the symlink resolution of concat, setQuoto, and getContentSummary
etc. Could you please add them?

I filed HDFS-944 for FileContext and AFS to provide the full ClientProtocol interface, currently
they're not reachable via FileContext or AFS. Since symlink resolution is partly implemented
in FileContext clients must be ported to FileContet to use them, ie if you create a symlink
via FileContext then access it via DistributedFileSystem#setQuota you'll get an UnresolvedLinkException.
Symlinks won't be exposed to users until we've moved off FileSystem (HADOOP-6446). We first
need to port (or rather have FileContext versions of) all the tests that haven't been covered
(HDFS-876 and HDFS-934).

I will update the design doc on the jira tomorrow to reflect the documentation I've put in
the jira comments, and the following. 

Patch attached here and to HADOOP-6421, addresses the above and some additional feedback from


> Create symbolic links in HDFS
> -----------------------------
>                 Key: HDFS-245
>                 URL: https://issues.apache.org/jira/browse/HDFS-245
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: dhruba borthakur
>            Assignee: Eli Collins
>         Attachments: 4044_20081030spi.java, designdocv1.txt, designdocv2.txt, designdocv3.txt,
HADOOP-4044-strawman.patch, symlink-0.20.0.patch, symlink-25-hdfs.patch, symlink-26-hdfs.patch,
symlink-26-hdfs.patch, symLink1.patch, symLink1.patch, symLink11.patch, symLink12.patch, symLink13.patch,
symLink14.patch, symLink15.txt, symLink15.txt, symlink16-common.patch, symlink16-hdfs.patch,
symlink16-mr.patch, symlink17-common.txt, symlink17-hdfs.txt, symlink18-common.txt, symlink19-common-delta.patch,
symlink19-common.txt, symlink19-common.txt, symlink19-hdfs-delta.patch, symlink19-hdfs.txt,
symlink20-common.patch, symlink20-hdfs.patch, symlink21-common.patch, symlink21-hdfs.patch,
symlink22-common.patch, symlink22-hdfs.patch, symlink23-common.patch, symlink23-hdfs.patch,
symlink24-hdfs.patch, symlink27-hdfs.patch, symlink28-hdfs.patch, symlink29-hdfs.patch, symlink29-hdfs.patch,
symlink30-hdfs.patch, symlink31-hdfs.patch, symlink33-hdfs.patch, symlink35-hdfs.patch, symlink36-hdfs.patch,
symLink4.patch, symLink5.patch, symLink6.patch, symLink8.patch, symLink9.patch
> HDFS should support symbolic links. A symbolic link is a special type of file that contains
a reference to another file or directory in the form of an absolute or relative path and that
affects pathname resolution. Programs which read or write to files named by a symbolic link
will behave as if operating directly on the target file. However, archiving utilities can
handle symbolic links specially and manipulate them directly.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message