hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Namit Jain (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3370) HDFS hardlink
Date Fri, 04 May 2012 17:40:49 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268564#comment-13268564
] 

Namit Jain commented on HDFS-3370:
----------------------------------

Another usecase in Hive is to copy one table/partition to another table/partition.
Ideally, we would like the following in Hive:

Copy Table T1 to T2.

The files under table location for T2 (say, /user/hive/warehouse/T2/0) can be a link to the
corresponding file in table T1
(say, /user/hive/warehouse/T1/0). 

Having said that, one of the requirements is that the data should be modified independently.
So, if a new data is loaded into T1 (or T2),
those changes should not be visible to T2 (or T1 respectively).
                
> HDFS hardlink
> -------------
>
>                 Key: HDFS-3370
>                 URL: https://issues.apache.org/jira/browse/HDFS-3370
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Hairong Kuang
>            Assignee: Liyin Tang
>
> We'd like to add a new feature to HDFS, hardlink, that allows harlinked files to share
data without copying. Currently we will support hardlinking only closed files, but it could
be extended to unclosed files as well.
> Among many potential use cases of feature, the following two are primarily used in facebook:
> 1. This provides a lightweight way for applications like hbase to create a snapshot;
> 2. This also allows an application like Hive to move a table to a different directory
without breaking current running hive queries.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message