hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3493) Add the default mapreduce.shuffle.port property to mapred-default.xml
Date Tue, 10 Apr 2012 04:29:37 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250449#comment-13250449

Harsh J commented on MAPREDUCE-3493:

Yes thats right, but it is also a service rather than an instance every reducer runs. So perhaps
a better description could be "Default port that the ShuffleHandler will run on. ShuffleHandler
is a service run at the NodeManager, to facilitate transfers of intermediate Map outputs to
requesting Reducers."
> Add the default mapreduce.shuffle.port property to mapred-default.xml
> ---------------------------------------------------------------------
>                 Key: MAPREDUCE-3493
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3493
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Ahmed Radwan
>            Assignee: Ahmed Radwan
>            Priority: Minor
>              Labels: newbie
>         Attachments: MAPREDUCE-3493-1.patch, MAPREDUCE-3493.patch
> I faced this issue when trying to run multiple Hadoop MR2 instances on the same node.
The default value for this property is hardcoded in the ShuffleHandler.java class so it results
in port conflicts. The issue is resolved if you set the property value in your conf files.
But the absence of this property from *-default.xml files is confusing. So It'll be cleaner
to move this property to mapred-default.xml, so its default value can be easily identified
and changed if needed. 

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message