hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3370) HDFS hardlink
Date Wed, 20 Jun 2012 18:04:43 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397700#comment-13397700

Konstantin Shvachko commented on HDFS-3370:

Sanjay, you are taking a quote out of context. It has been explained what "hard" means above.
Please scan through. One more example:
Well understood why traditional hard links are not allowed across volumes. A distributed namespace
is like dynamically changing volumes. You can restrict a link to a single volume, but the
names can flow to different volumes later on.

I am not proposing to remove the existing complexity from the system, I propose not to introduce
more of it. In distributed case consistent hard links need PAXOS-like algorithms. They are
not "elementary operations", which only should compose the API.
Hard links can be implemented as a library using ZK, which will stand in distributed case.

A couple of quotes from Snajay's (mine too) favorite author:
- When in doubt, leave it out. If there is a fundamental theorem of API design, this is it.
You can always add things later, but you can't take them away.
- APIs must coexist peacefully with the platform, so do what is customary. It is almost always
wrong to transliterate an API from one platform to another.
- Consider the performance consequences of API design decisions ...
> HDFS hardlink
> -------------
>                 Key: HDFS-3370
>                 URL: https://issues.apache.org/jira/browse/HDFS-3370
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Hairong Kuang
>            Assignee: Liyin Tang
>         Attachments: HDFS-HardLink.pdf
> We'd like to add a new feature hardlink to HDFS that allows harlinked files to share
data without copying. Currently we will support hardlinking only closed files, but it could
be extended to unclosed files as well.
> Among many potential use cases of the feature, the following two are primarily used in
> 1. This provides a lightweight way for applications like hbase to create a snapshot;
> 2. This also allows an application like Hive to move a table to a different directory
without breaking current running hive queries.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message