hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-1533) Distcp should log to specified location
Date Sun, 01 Jul 2007 06:08:04 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Owen O'Malley updated HADOOP-1533:

    Status: Open  (was: Patch Available)

This patch seems confused whether the "log path" is a Path, URI, or a String. Since you are
implementing it as the output of the Map (with reduces = 0), it pretty clearly must be a Path.
It should be coded as such.

Instead of requiring a parameter for the log path, I think it should be an option like:
  ... - log hdfs://server:1234/my/log
and I'd propose that the default be: "$output_dir/_distcp_logs"

That will work well in most cases and keep the output/logs together.

The chunk of code that looks like:

+    String logURL = "hdfs://" + fsConfig.get("fs.default.name", "local") + 
+                         archiveDirectory + "distcpLogDir." + System.currentTimeMillis();

is problematic, since it won't construct useful urls in a wide variety of values for fs.default.name.

> Distcp should log to specified location
> ---------------------------------------
>                 Key: HADOOP-1533
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1533
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: util
>            Reporter: Senthil Subramanian
>             Fix For: 0.14.0
>         Attachments: HADOOP-1533_1.patch
> Distcp errors are now not logged anywhere (other that reporter.setStatus which gets overwritten).
Add support to mention a log URI where the log files should go.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message