Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Mon, 25 Feb 2013 23:14:14 +0000 (UTC)
From: "Aaron T. Myers (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12633724.1361583507831.337364.1361834054557@arcas>
In-Reply-To: <JIRA.12633724.1361583507831@arcas>
References: <JIRA.12633724.1361583507831@arcas>
Subject: [jira] [Commented] (HDFS-4523) Fix concat for snapshots
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13586440#comment-13586440 ] 

Aaron T. Myers commented on HDFS-4523:
--------------------------------------

bq. The original files has to be set up specifically for concat. It is not like that you can concat on any set of files.

I realize that, but I don't see that as a good reason to treat them differently in the context of a read-only, immutable snapshot.

bq. On the other hand, we may fail concat if the transient files are in some snapshots.

Why couldn't we both retain the source files in the snapshots and allow the operation to succeed, producing the target file as normal in the present file system? That's the behavior I would expect out of a system which supports read-only snapshots.
                
> Fix concat for snapshots
> ------------------------
>
>                 Key: HDFS-4523
>                 URL: https://issues.apache.org/jira/browse/HDFS-4523
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>            Reporter: Tsz Wo (Nicholas), SZE
>            Assignee: Tsz Wo (Nicholas), SZE
>         Attachments: h4523_20130222.patch, h4523_20130223.patch, h4523_20130225.patch
>
>
> The use case of concat is for copying large files across clusters using the following steps.
> - Step 1: The blocks of a file in the source cluster are copied in parallel to transient files in the destination cluster.
> - Step 2: Then the transient files in the destination cluster are concatenated in order to obtain the original file.
> If a snapshot is taken in the destination cluster before Step 2, some transient files may be captured in the snapshot.  These transient files should be removed in Step 2.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira