hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravi Gummadi (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-5447) Logging turned on by default while using distcp
Date Tue, 16 Jun 2009 09:59:07 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ravi Gummadi updated HADOOP-5447:
---------------------------------

    Attachment: d_log_5447.patch

Without -log option and with relative path specified   to -log, the logs dir distcp_logs_randomID
was created in copy destination dir.

Attaching patch that makes distcp to create distcp_logs_randomID dir in mapred.system.dir
and not in the copy destination dir. Also displays a message giving the path of this log dir
for users to look into its contents, if needed. Since the log dir name contains randomID as
part of it, multiple distcp jobs shouldn't get the same log dir name, thus avoiding clashing.

Please review and provide your comments.

> Logging turned on by default while using distcp
> -----------------------------------------------
>
>                 Key: HADOOP-5447
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5447
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: tools/distcp
>    Affects Versions: 0.18.2, 0.18.3
>         Environment: Linux RHEL54, Linux RHEL5
>            Reporter: Sanjeev Jaiswal
>            Assignee: Ravi Gummadi
>             Fix For: 0.20.1
>
>         Attachments: d_log_5447.patch
>
>
> Distcp should have an option to disable logging during a distcp
> eg: hadoop distcp --nolog <source dir> <destination dir>
> By default logging is enabled or turned on while using distcp.This generates logs in
DFS, which need to be cleaned up periodically. During this time, critical applications sometimes
fail, when they see distcp logs in the DFS.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message