hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Joseph Evans (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2652) MR-279: Cannot run multiple NMs on a single node
Date Mon, 29 Aug 2011 21:57:37 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093247#comment-13093247

Robert Joseph Evans commented on MAPREDUCE-2652:


I think I have the new patch ready, I just want to clarify a few things before going forward
with it.
This clearly seems to tell us (:)) that we need a subclass of TaskAttemptEvent - TaskAttemptContainerLaunchedEvent.
We can put the shuffle-port as a payload of that event. Once you do that, the rest of the
changes to TaskAttempt.java can be dropped.

I added in the subclass but I am not sure what changes to TaskAttempt.java can be dropped.
 Did you mean the other changes to TaskAttemptEvent.java can be dropped?

The TODO should have been deleted a while ago.  It was there before I understood the code,
and I was just marking where similar data was being used.  Do you think I need to add something
to the TaskCompletion events returned to the client?

> MR-279: Cannot run multiple NMs on a single node 
> -------------------------------------------------
>                 Key: MAPREDUCE-2652
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2652
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Robert Joseph Evans
>            Assignee: Robert Joseph Evans
>             Fix For: 0.23.0
>         Attachments: MR-2652-trunk-v3.patch, MR-2652-v1.txt, MR-2652-v2.txt
> Currently in MR-279 the Auxiliary services, like ShuffleHandler, have no way to communicate
information back to the applications.  Because of this the Map Reduce Application Master has
hardcoded in a port of 8080 for shuffle.  This prevents the configuration "mapreduce.shuffle.port"
form ever being set to anything but 8080.  The code should be updated to allow this information
to be returned to the application master.  Also the data needs to be persisted to the task
log so that on restart the data is not lost.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message