hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Lucene-hadoop Wiki] Update of "HowToDebugMapReducePrograms" by Amareshwari
Date Mon, 01 Oct 2007 05:14:29 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.

The following page has been changed by Amareshwari:
http://wiki.apache.org/lucene-hadoop/HowToDebugMapReducePrograms

------------------------------------------------------------------------------
  
  == Run a debug script when Task fails ==
  
- A facility is provided, via user-provided scripts, for doing post-processing on task logs,
task's stdout, stderr, core file. There is a default script which processes core dumps under
gdb and prints stack trace. The last five lines from stdout and stderr of debug script are
printed on the diagnostics. These outputs are displayed on job UI on demand. 
+ A facility is provided, via user-provided scripts, for doing post-processing on task logs,
task's stdout, stderr, syslog and core files. There is a default script which processes core
dumps under gdb and prints stack trace. The last five lines from stdout and stderr of debug
script are printed on the diagnostics. These outputs are displayed on job UI on demand. 
  
  == How to submit debug command ==
  
  A quick way to set debug command is to set the properties "mapred.map.task.debug.command"
and "mapred.reduce.task.debug.command" for debugging map task and reduce task respectively.
  These properties can also be set by APIs conf.setMapDebugCommand(String cmd) and conf.setReduceDebugCommand(String
cmd).
- The debug command can consist of @stdout@, @stderr@, @core@ to access task's stdout, stderr
and core files respectively.
+ The debug command can consist of @stdout@, @stderr@, @syslog@ and @core@ to access task's
stdout, stderr syslog and core files respectively.
  In case of streaming debug command can be submitted with command-line options -mapdebug,
-reducedebug for debugging mapper and redcuer respectively.
  
- For examples command can be 'myScript @stderr@'. This command has executable myScript. And
myScript processes failed task's stderr.
+ For example, the debug command can be 'myScript @stderr@'. This command has executable myScript.
And myScript processes failed task's stderr.
  
  The debug command can be a gdb command where user can submit a command file to execute using
-x option. 
  Then debug command can look like 'gdb <program-name> -c @core@ -x <gdb-cmd-fle>
'. This command processes core file of the failed task <program-name> and executes commands
in <gdb-cmd-file>. Please make sure gdb command file has 'quit' in its last line.

Mime
View raw message