hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-2178) Job history on HDFS
Date Thu, 24 Jan 2008 05:12:34 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561944#action_12561944

Devaraj Das commented on HADOOP-2178:

Some comments (sorry for being late on this)
1) Some of the configurable items are not listed in hadoop-default.xml. For example - mapred.job.history.http.bindAddress
2) mapred.job.history.viewer seems redundant
3) The documentation should call out those cases, where history files are getting created
in the output directory, explicitly. Something along the lines of having to write a filter
for listing the output directory esp for cases where the output dir would be consumed by a
subsequent MR job. I think it makes sense to have the filter be prepackaged as part of hadoop.
That way every user doesn't have to implement the same thing.
4) The documentation should mention that the user should start a browser and connect to the
host:port that he gets when he starts the viewer.

> Job history on HDFS
> -------------------
>                 Key: HADOOP-2178
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2178
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Amareshwari Sri Ramadasu
>            Assignee: Amareshwari Sri Ramadasu
>             Fix For: 0.16.0
>         Attachments: patch-2178.txt, patch-2178.txt, patch-2178.txt, patch-2178.txt,
patch-2178.txt, patch-2178.txt
> This issue addresses the following items :
> 1.  Check for accuracy of job tracker history logs.
> 2.  After completion of the job, copy the JobHistory.log(Master index file) and the job
history files to the DFS.
> 3. User can load the history with commands
> bin/hadoop job -history <directory> 
> or
> bin/hadoop job -history <jobid>
> This will start a stand-alone jetty and load jsps

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message