hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4044) Create symbolic links in HDFS
Date Thu, 11 Sep 2008 08:11:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630123#action_12630123
] 

dhruba borthakur commented on HADOOP-4044:
------------------------------------------

@Raghu: The FSDataInputStream will implement another interface StreamType that will store
the information on whether this is a symbolc link. No extra RPC is needed.

@Doug: By definition, the filesystem has no control about the target of a symbolic link. The
user could make it point to a non-existent path too. There will be no enforcement of whether
the target is a directory or not.

@Allen: This proposal should not introduce additional grief for namenode. If you replace a
file with a symbolic link, it still occupies one inode. No additional inodes are necessary.
I agree that admins might like symbolic links to directories, but maybe we can do that at
a later date when the user-case becomes more concrete. Does it sound acceptable?

The simplicity for the filesystem is that no "kickbacks" while traversing paths are needed.

{noformat}

  /**
   * Opens an FSDataInputStream at the indicated Path.
   * This does not follow symbolic links.
   * @param f the file name to open
   * @param bufferSize the size of the buffer to be used.
   */
  public abstract FSDataInputStream openfs(Path f, int bufferSize)
    throws IOException;

  /**
   * Opens an FSDataInputStream at the indicated Path.
   * If the specified path is a symbolic link, then open the
   * target of the symbolic link.
   * @param f the file to open
   */
  public FSDataInputStream open(Path f) throws IOException {
    return open(f, getConf().getInt("io.file.buffer.size", 4096));
  }

  /**
   * Opens an FSDataInputStream at the indicated Path.
   * @param f the file to open
   * @param bufferSize the size of the buffer to be used.
   */
  public FSDataInputStream open(Path f, int bufferSize)
    throws IOException {
    FileSystem fs = this;
    FSDataInputStream in = fs.openfs(f, bufferSize);
    while (in.isSymlink()) {
      // construct new path pointed to by symlink
      Path newpath = new Path(in.getSymlink());
      if (!newpath.isAbsolute()) {
        newpath = new Path(f.getParent(), newpath);
      }
      in.close();
      f = newpath;
      fs = f.getFileSystem(getConf());
      LOG.warn("XXX Opening symlink at " + f);
      in = fs.openfs(f, bufferSize);
    }
    return in;
  }


public class FSDataInputStream extends DataInputStream
    implements Seekable, PositionedReadable, StreamType {
....
}


/** Types of streams */
interface StreamType {
  /**
   * Is this stream a symbolic link?
   */
  public boolean isSymlink() throws IOException;

  /**
   * Return the contents of the symlink
   */
  public String getSymlink() throws IOException;
}

{noformat}

> Create symbolic links in HDFS
> -----------------------------
>
>                 Key: HADOOP-4044
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4044
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>         Attachments: symLink1.patch
>
>
> HDFS should support symbolic links. A symbolic link is a special type of file that contains
a reference to another file or directory in the form of an absolute or relative path and that
affects pathname resolution. Programs which read or write to files named by a symbolic link
will behave as if operating directly on the target file. However, archiving utilities can
handle symbolic links specially and manipulate them directly.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message