hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3370) HDFS hardlink
Date Thu, 28 Jun 2012 00:56:43 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13402735#comment-13402735
] 

Daryn Sharp commented on HDFS-3370:
-----------------------------------

Nice idea, but I think it gets much more complicated.  Retaining ref-counted paths after deletion
in the origin namespace requires an "inode id".  A new api to reference paths based on the
id is required.  We aren't so soft anymore...

The inode id needs to be secured since it bypasses all parent dir permissions, yet the id
should be identical for all links in order for copy utils to distinguish identical inodes.

Now comes the worst part: the client.  Will the NNs proxy fs stream operations to each other
with a secure api for referencing inode ids?  Or will they redirect the client to the origin
NN?  If they redirect, how to protect against the client guessing ids, or saving them for
later replay even when the dir privs prevent access?
                
> HDFS hardlink
> -------------
>
>                 Key: HDFS-3370
>                 URL: https://issues.apache.org/jira/browse/HDFS-3370
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Hairong Kuang
>            Assignee: Liyin Tang
>         Attachments: HDFS-HardLink.pdf
>
>
> We'd like to add a new feature hardlink to HDFS that allows harlinked files to share
data without copying. Currently we will support hardlinking only closed files, but it could
be extended to unclosed files as well.
> Among many potential use cases of the feature, the following two are primarily used in
facebook:
> 1. This provides a lightweight way for applications like hbase to create a snapshot;
> 2. This also allows an application like Hive to move a table to a different directory
without breaking current running hive queries.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message