hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sri Ramadasu (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (HADOOP-1857) Ability to run a script when a task fails to capture stack traces
Date Thu, 11 Oct 2007 12:14:51 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534016
] 

amareshwari edited comment on HADOOP-1857 at 10/11/07 5:13 AM:
----------------------------------------------------------------------------

The patch attached incorparates the comments.

Changes done in this patch are

1. The command has single interface:  $script $stdout $stderr $syslog $jobconf
2. Adding executables is removed. And all files coming out of distributed cache have executation
permission.
3. Code for finding core file is removed. And default script for pipes will do it in the script.
4. Both stdout and stdin of debug script are directed to debugout.
5. Everything in debugout is added in diagnostics.

Usage documentation is updated at http://wiki.apache.org/lucene-hadoop/HowToDebugMapReducePrograms



      was (Author: amareshwari):
    
The patch attached incorparates the comments.

Changes done in this patch are

1. The command has single interface $script $stdout $stderr $syslog
2. Adding executables is removed. And all files coming out of distributed cache have executation
permission.
3. Code for finding core file is removed. And default script for pipes will do it in the script.
4. Both stdout and stdin of debug script are directed to debugout.
5. Everything in debugout is added in diagnostics.

Usage documentation is updated at http://wiki.apache.org/lucene-hadoop/HowToDebugMapReducePrograms


  
> Ability to run a script when a task fails to capture stack traces
> -----------------------------------------------------------------
>
>                 Key: HADOOP-1857
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1857
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.14.0
>            Reporter: Amareshwari Sri Ramadasu
>            Assignee: Amareshwari Sri Ramadasu
>         Attachments: patch-1857.txt, patch-1857.txt, patch-1857.txt, patch-1857.txt,
patch-1857.txt, patch-1857.txt, patch1857.txt, tt-no-warn.patch
>
>
> This basically is for providing a better user interface for debugging failed
> jobs. Today we see stack traces for failed tasks on the job ui if the job
> happened to be a Java MR job. For non-Java jobs like Streaming, Pipes, the
> diagnostic info on the job UI is not helpful enough to debug what might have
> gone wrong. They are usually framework traces and not app traces.
> We want to be able to provide a facility, via user-provided scripts, for doing
> post-processing on task logs, input, output, etc. There should be some default
> scripts like running core dumps under gdb for locating illegal instructions,
> the last few lines from stderr, etc.  These outputs could be sent to the
> tasktracker and in turn to the jobtracker which would then display it on the
> job UI on demand.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message