hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liyin Tang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3370) HDFS hardlink
Date Wed, 09 May 2012 06:18:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271141#comment-13271141
] 

Liyin Tang commented on HDFS-3370:
----------------------------------

bq. Another consideration is ds quota is based on a multiple of replication factor, so who
is allowed to change the replication factor since increasing it may impact a different user's
quota?

Generally, when user creates a hardlink in Linux, it requires the EXECUTE permission for the
source directory and WRITE_EXECUTE permission for the destination directory. And it is a well-known
issue that hard links on Linux could create local DoS vulnerability and security problems,
especially when malicious user keeps creating hard links to other users files and let others
run out of quota. One of solutions to prevent this problem is to set the permission of the
dir correctly.

HDFS hardlink should follow the same permission requirements as genreal Linux FS and only
allow the trusted users or groups have right permission to create hardlinks. The same security
principle shall apply for setReplication operation, which can be treated as a normal write
operation in general Linux FS.

Thanks Daryn Sharp so much for the above discussion. 
It really helps us to re-visit several design issues and improve the solutions. I will update
the design doc later. 
                
> HDFS hardlink
> -------------
>
>                 Key: HDFS-3370
>                 URL: https://issues.apache.org/jira/browse/HDFS-3370
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Hairong Kuang
>            Assignee: Liyin Tang
>
> We'd like to add a new feature hardlink to HDFS that allows harlinked files to share
data without copying. Currently we will support hardlinking only closed files, but it could
be extended to unclosed files as well.
> Among many potential use cases of the feature, the following two are primarily used in
facebook:
> 1. This provides a lightweight way for applications like hbase to create a snapshot;
> 2. This also allows an application like Hive to move a table to a different directory
without breaking current running hive queries.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message