hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Joseph Evans (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2652) MR-279: Cannot run multiple NMs on a single node
Date Tue, 30 Aug 2011 13:55:38 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093738#comment-13093738
] 

Robert Joseph Evans commented on MAPREDUCE-2652:
------------------------------------------------

Great.  I will have the patch uploaded very soon.  By changing the TaskAttemptEvent to a different
class it exposed that I had not handled the start container event consistently everywhere
it was sent.  I missed app master recovery, local mode, and MRApp test mode.  So I have updated
them all.

Also the unit test changes you wanted are more stubbed out then I expected.  MRApp uses a
MockContainerLauncher, which does not really launch anything.  It does not talk to the TaskTracker
so setting the config value will not really test anything I changed.  I reverted my change
to the test because of that.  I have tested it manually and I can see how it would be nice
to have a unit test to verify that, but that would require some deep changes to MRApp to make
that work.

> MR-279: Cannot run multiple NMs on a single node 
> -------------------------------------------------
>
>                 Key: MAPREDUCE-2652
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2652
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Robert Joseph Evans
>            Assignee: Robert Joseph Evans
>             Fix For: 0.23.0
>
>         Attachments: MR-2652-trunk-v3.patch, MR-2652-v1.txt, MR-2652-v2.txt
>
>
> Currently in MR-279 the Auxiliary services, like ShuffleHandler, have no way to communicate
information back to the applications.  Because of this the Map Reduce Application Master has
hardcoded in a port of 8080 for shuffle.  This prevents the configuration "mapreduce.shuffle.port"
form ever being set to anything but 8080.  The code should be updated to allow this information
to be returned to the application master.  Also the data needs to be persisted to the task
log so that on restart the data is not lost.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message