hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1699) Child task debugging on Hadoop
Date Thu, 09 Aug 2007 03:27:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12518600

Arun C Murthy commented on HADOOP-1699:

Please take a look at the config knob {{mapred.child.java.opts}}:


  <description>Java opts for the task tracker child processes.  Subsumes
  'mapred.child.heap.size' (If a mapred.child.heap.size value is found
  in a configuration, its maximum heap size will be used and a warning
  emitted that heap.size has been deprecated). Also, the following symbols,
  if present, will be interpolated: @taskid@ is replaced by current TaskID;
  and @port@ will be replaced by mapred.task.tracker.report.port + 1 (A second
  child will fail with a port-in-use if mapred.tasktracker.tasks.maximum is
  greater than one). Any other occurrences of '@' will go unchanged. For
  example, to enable verbose gc logging to a file named for the taskid in
  /tmp and to set the heap maximum to be a gigabyte, pass a 'value' of:

        -Xmx1024m -verbose:gc -Xloggc:/tmp/@taskid@.gc


I'm inclined to resolve this bug as *Invalid* since it already is supported... agree?

> Child task debugging on Hadoop
> ------------------------------
>                 Key: HADOOP-1699
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1699
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>         Environment: Any hadoop installation
>            Reporter: Srikanth Kakani
>            Priority: Critical
> Recently I discovered debugging a map/reduce task on hadoop can be simplified with couple
of lines code in TaskRunner enabling us to run jdb on the child tasks.
> TaskRunner.java: 282
>         if(conf.getBoolean("mapred.debug.child.task", false)) {
>             Random r=new Random();
>             int debugPort=8000 + r.nextInt(1000);
>             vargs.add("-Xdebug");
>             vargs.add("-Xrunjdwp:transport=dt_socket,address="+ debugPort +",server=y,suspend=n");
>             LOG.info("Running debug server for task "+ t.getTaskId() + " at port" + debugPort);
>         }
> This code runs the child tasks with debug enabled after getting a configuration variable.
> Connecting to a child task is as simple as running jdb -attach <hostname>:<debugPort>
from anywhere. Additionally authentication information could be included in the jobConf. 
> I believe it will greatly reduce the development/debug time on hadoop.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message