hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yongjun Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7312) Update DistCp v1 to optionally not use tmp location
Date Mon, 24 Nov 2014 18:15:12 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14223234#comment-14223234
] 

Yongjun Zhang commented on HDFS-7312:
-------------------------------------

Hi Joe, 

Thanks for addressing my earlier comments, and the various testing you have done. The patch
looks good to me,  you may consider addressing the following nits later after getting committer's
review:

* line 402 is mis-aligned.
* Consolidate {{testCopyFromDfsToLocalWithStagingAreaInDestSkiptmp}} with {{testCopyFromDfsToLocalWithStagingAreaInDest}}
to share the same code, like other tests changed in the patch.



> Update DistCp v1 to optionally not use tmp location
> ---------------------------------------------------
>
>                 Key: HDFS-7312
>                 URL: https://issues.apache.org/jira/browse/HDFS-7312
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: tools
>    Affects Versions: 2.5.1
>            Reporter: Joseph Prosser
>            Assignee: Joseph Prosser
>            Priority: Minor
>         Attachments: HDFS-7312.001.patch, HDFS-7312.002.patch, HDFS-7312.003.patch, HDFS-7312.004.patch,
HDFS-7312.005.patch, HDFS-7312.006.patch, HDFS-7312.007.patch, HDFS-7312.patch
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> DistCp v1 currently copies files to a tmp location and then renames that to the specified
destination.  This can cause performance issues on filesystems such as S3.  A -skiptmp flag
will be added to bypass this step and copy directly to the destination.  This feature mirrors
a similar one added to HBase ExportSnapshot [HBASE-11119|https://issues.apache.org/jira/browse/HBASE-11119]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message