hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2637) Providing options to debug the mapreduce user code (Mapper, Reducer, Combiner, Sort implementations)
Date Tue, 12 Jul 2011 06:56:00 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13063749#comment-13063749
] 

Arun C Murthy commented on MAPREDUCE-2637:
------------------------------------------

I'm not sure this is a scalable solution, do we really want each map/reduce task to come up
with a server for connecting to - particularly for large jobs this makes very little sense.

OTOH, most profilers allow you to 'dump' the profile information to the local disk. Hadoop
MR also has support for automatically shipping that to the job-client. Doesn't that suffice?

> Providing options to debug the mapreduce user code (Mapper, Reducer, Combiner, Sort implementations)
> ----------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2637
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2637
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: tasktracker
>            Reporter: Devaraj K
>            Assignee: Devaraj K
>
> Presently Hadoop provides "mapred.child.java.opts" configuration which can be used to
set JVM options for Child JVM running Map or Reduce Task. 
> If we need to remote debug the Child JVM, we can add remote debugging options to this
configuration value.
> But this will work only for single Child JVM. Other children will fail as the remote
debugging port is already used.
> We cannot specify the remote debugging port dynamically.
> As a result, it's not possible to remote debug multiple Child JVMs.
> As a solution to this problem, we can provide a configuration to debug Task JVMs in this
scenario.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message