hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Enis Soztutar (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-53) MapReduce log files should be storable in dfs.
Date Thu, 18 Oct 2007 15:37:50 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-53?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Enis Soztutar updated HADOOP-53:

    Attachment: mapredDFSLog_v1.patch

Attaching a patch to store the tasks' logs to the FileSystem used. This is useful for example
to store the logs permanently or to access the logs in a centralized way. 

The patch adds a new log4j Appender, called FsLogAppender, which appends logs to the files
in the FileSystem. The appender is called FSLA. The old appender(TLA) still continues. The
user can select which appender to use, and which log level to use via JobConf(). ex : job.setTaskLogRootLogger("INFO,TLA,FSLA");

The user can also specify the location to save the logs : job.setTaskLogDir(Path);

Now, at DEBUG level the logs from org.apache.hadoop will continue to polute the logs of the
user's program, but we will defer this to a separate issue. 

The appender can also be used to store the logs of the framework(for debugging etc,), but
again it is a seperate issue. 

> MapReduce log files should be storable in dfs.
> ----------------------------------------------
>                 Key: HADOOP-53
>                 URL: https://issues.apache.org/jira/browse/HADOOP-53
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: mapred
>    Affects Versions: 0.2.0
>            Reporter: Doug Cutting
>            Assignee: Enis Soztutar
>         Attachments: mapredDFSLog_v1.patch
> It should be possible to cause a job's log output to be stored in dfs.  The jobtracker's
log output and (optionally) all tasktracker log output related to a job should be storable
in a job-specified dfs directory.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message