hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <Jason_Vasd...@McAfee.com>
Subject FW: meaning of cryptic yarn nodemanager message: org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices
Date Wed, 13 Nov 2013 12:49:25 GMT
Good day - please could anyone explain the meaning of this message in yarn-${USER}-nodemanager-${HOST}.log:

"
2013-11-13 12:17:29,704 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices:
The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class class org.apache.hadoop.mapred.ShuffleHandler
which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData
and read Service Meta Data may have issues unless the refer to the name in the config.
"
Love the creative spelling ! ('Auxilurary' should be 'Auxiliary' ? 'unless the refer' should
be 'unless they refer' ?)

I guess this is because of the setting in yarn-site.xml :

"
<property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
    <description>shuffle service that needs to be set for Map Reduce to run </description>
 </property> "
that I installed according to the instructions at:  
http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-common/SingleCluster.html

Why do these instructions specify this setting it may cause "issues" ? What sort of issues
may be caused ? 
This sentence is nonsensical:
  " Because these are not the same tools trying to send ServiceData and read Service Meta
Data may have issues unless the refer to the name in the config."
What are the "tools" being referred to here ?
What "may have issues" ?
What are the "issues" ? 
What  must refer to the name in the config ?
Should I change the name to "httpshuffle"  or not ?

Any ideas / suggestions gratefully received.

Regards,
Jason

Mime
View raw message