hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Frederick Tucker (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-6734) Add option to distcp to preserve file path structure of source files at the destination
Date Mon, 18 Jul 2016 18:54:20 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-6734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Frederick Tucker updated MAPREDUCE-6734:
----------------------------------------
    Status: Patch Available  (was: Open)

Added -preservepath and -sourceprefixmask options to distcp to control how to the filestructure
is copied to target fs when using distcp. 

-preservepath: Preserve the absolute path of the source file at the target.
-sourceprefixmas: Remove the start of a source's absolute path when running distcp with -preservepath.

Tests, javadoc, and wiki are updated as well.

> Add option to distcp to preserve file path structure of source files at the destination
> ---------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6734
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6734
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: distcp
>    Affects Versions: 3.0.0-alpha2
>         Environment: Software platform
>            Reporter: Frederick Tucker
>            Priority: Critical
>              Labels: distcp, newbie, patch
>             Fix For: 3.0.0-alpha2
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> When copying files using distcp with globbed source files, all the matched files in the
glob are copied in a single flat directory.  This causes problems when the file structure
at the source is important.  It also is an issue when there are two files matched in the glob
with the same name because it causes a duplicate file error at the target.  I'd like to have
an option to preserve the file structure of the source files when globbing inputs.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message