hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prashant Sharma (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-3211) Fetch failures if used ephemeral port for property "mapreduce.shuffle.port".
Date Wed, 19 Oct 2011 09:50:11 GMT
Fetch failures if used ephemeral port for property "mapreduce.shuffle.port". 
-----------------------------------------------------------------------------

                 Key: MAPREDUCE-3211
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3211
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: job submission, mrv2
    Affects Versions: 0.24.0
         Environment: Linux
            Reporter: Prashant Sharma
            Assignee: Prashant Sharma
             Fix For: 0.24.0


To reproduce the bug. 

Use the following property in mapred-site.xml

<configuration>
    <property>
      <name> mapreduce.framework.name</name>
      <value>yarn</value>
    </property>
    <property>
      <name> mapreduce.shuffle.port</name>
      <value>0</value>
    </property>
</configuration>

and following in yarn-site.xml


  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce.shuffle</value>
    <description>shuffle service that needs to be set for Map Reduce to run </description>
  </property>

 <property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>

Then try to start the yarn daemons.

And submit the job. (which would eventually fail).

Also observe NM logs. It says shuffle handler service started at port 0; :P

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message