hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4044) Create symbolic links in HDFS
Date Thu, 11 Sep 2008 08:11:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630123#action_12630123

dhruba borthakur commented on HADOOP-4044:

@Raghu: The FSDataInputStream will implement another interface StreamType that will store
the information on whether this is a symbolc link. No extra RPC is needed.

@Doug: By definition, the filesystem has no control about the target of a symbolic link. The
user could make it point to a non-existent path too. There will be no enforcement of whether
the target is a directory or not.

@Allen: This proposal should not introduce additional grief for namenode. If you replace a
file with a symbolic link, it still occupies one inode. No additional inodes are necessary.
I agree that admins might like symbolic links to directories, but maybe we can do that at
a later date when the user-case becomes more concrete. Does it sound acceptable?

The simplicity for the filesystem is that no "kickbacks" while traversing paths are needed.


   * Opens an FSDataInputStream at the indicated Path.
   * This does not follow symbolic links.
   * @param f the file name to open
   * @param bufferSize the size of the buffer to be used.
  public abstract FSDataInputStream openfs(Path f, int bufferSize)
    throws IOException;

   * Opens an FSDataInputStream at the indicated Path.
   * If the specified path is a symbolic link, then open the
   * target of the symbolic link.
   * @param f the file to open
  public FSDataInputStream open(Path f) throws IOException {
    return open(f, getConf().getInt("io.file.buffer.size", 4096));

   * Opens an FSDataInputStream at the indicated Path.
   * @param f the file to open
   * @param bufferSize the size of the buffer to be used.
  public FSDataInputStream open(Path f, int bufferSize)
    throws IOException {
    FileSystem fs = this;
    FSDataInputStream in = fs.openfs(f, bufferSize);
    while (in.isSymlink()) {
      // construct new path pointed to by symlink
      Path newpath = new Path(in.getSymlink());
      if (!newpath.isAbsolute()) {
        newpath = new Path(f.getParent(), newpath);
      f = newpath;
      fs = f.getFileSystem(getConf());
      LOG.warn("XXX Opening symlink at " + f);
      in = fs.openfs(f, bufferSize);
    return in;

public class FSDataInputStream extends DataInputStream
    implements Seekable, PositionedReadable, StreamType {

/** Types of streams */
interface StreamType {
   * Is this stream a symbolic link?
  public boolean isSymlink() throws IOException;

   * Return the contents of the symlink
  public String getSymlink() throws IOException;


> Create symbolic links in HDFS
> -----------------------------
>                 Key: HADOOP-4044
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4044
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>         Attachments: symLink1.patch
> HDFS should support symbolic links. A symbolic link is a special type of file that contains
a reference to another file or directory in the form of an absolute or relative path and that
affects pathname resolution. Programs which read or write to files named by a symbolic link
will behave as if operating directly on the target file. However, archiving utilities can
handle symbolic links specially and manipulate them directly.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message