hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shaposhnikov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10234) DistCp log output should contain copied and deleted files and directories
Date Fri, 01 Apr 2016 06:34:25 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15221240#comment-15221240

Konstantin Shaposhnikov commented on HDFS-10234:

I guess my problem description was not very precise.

The log file that I am interested in is created when `-log logDir` option is specified. This
file is written using {{context.write}} (e.g. {{context.write(null, new Text("SKIP: " + sourceCurrStatus.getPath()));}}).

Currently this file contains only SKIP entries making it almost useless. 

There is no need to make logging on INFO level more verbose as it is already too verbose.

> DistCp log output should contain copied and deleted files and directories
> -------------------------------------------------------------------------
>                 Key: HDFS-10234
>                 URL: https://issues.apache.org/jira/browse/HDFS-10234
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: distcp
>    Affects Versions: 2.7.1
>            Reporter: Konstantin Shaposhnikov
>            Assignee: Lin Yiqun
>         Attachments: HDFS-10234.001.patch
> DistCp log output (specified via {{-log}} command line option) currently contains only
skipped and failed (when failures are ignored via {{-i}}) files.
> It will be more useful if it also contains copied and deleted files and created directories.
> This should be fixed in https://github.com/apache/hadoop/blob/branch-2.7.1/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java

This message was sent by Atlassian JIRA

View raw message