hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6152) distcp V2 doesn't preserve root dir's attributes when -p is specified
Date Wed, 02 Apr 2014 04:45:17 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13957331#comment-13957331

Hadoop QA commented on HDFS-6152:

{color:green}+1 overall{color}.  Here are the results of testing the latest attachment 
  against trunk revision .

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 10 new or modified
test files.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

    {color:green}+1 eclipse:eclipse{color}.  The patch built with eclipse:eclipse.

    {color:green}+1 findbugs{color}.  The patch does not introduce any new Findbugs (version
1.3.9) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:green}+1 core tests{color}.  The patch passed unit tests in hadoop-tools/hadoop-distcp.

    {color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6570//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6570//console

This message is automatically generated.

> distcp V2 doesn't preserve root dir's attributes when -p is specified
> ---------------------------------------------------------------------
>                 Key: HDFS-6152
>                 URL: https://issues.apache.org/jira/browse/HDFS-6152
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs-client
>    Affects Versions: 2.3.0
>            Reporter: Yongjun Zhang
>            Assignee: Yongjun Zhang
>         Attachments: HDFS-6152.001.patch, HDFS-6152.002.patch, HDFS-6152.002.patch, HDFS-6152.003.patch
> Two issues were observed with distcpV2
> ISSUE 1. when copying a source dir to target dir with "-pu" option using command 
>   "distcp -pu source-dir target-dir"
> The source dir's owner is not preserved at target dir. Simiarly other attributes of source
dir are not preserved.  Supposedly they should be preserved when no -update and no -overwrite
> There are two scenarios with the above command:
> a. when target-dir already exists. Issuing the above command will  result in target-dir/source-dir
(source-dir here refers to the last component of the source-dir path in the command line)
at target file system, with all contents in source-dir copied to under target-dir/src-dir.
The issue in this case is, the attributes of src-dir is not preserved.
> b. when target-dir doesn't exist. It will result in target-dir with all contents of source-dir
copied to under target-dir. This issue in this  case is, the attributes of source-dir is not
carried over to target-dir.
> For multiple source cases, e.g., command 
>   "distcp -pu source-dir1 source-dir2 target-dir"
> No matter whether the target-dir exists or not, the multiple sources are copied to under
the target dir (target-dir is created if it didn't exist). And their attributes are preserved.

> ISSUE 2. with the following command:
>   "distcp source-dir target-dir"
> when source-dir is an empty directory, and when target-dir doesn't exist, source-dir
is not copied, actually the command behaves like a no-op. However, when the source-dir is
not empty, it would be copied and results in target-dir at the target file system containing
a copy of source-dir's children.
> To be consistent, empty source dir should be copied too. Basically the  above distcp
command should cause target-dir get created at target file system, and the source-dir's attributes
are preserved at target-dir when -p is passed.

This message was sent by Atlassian JIRA

View raw message