hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Lilley <john.lil...@redpoint.net>
Subject NM/AM interaction
Date Tue, 28 May 2013 22:43:16 GMT
I was reading from the HortonWorks blog:
"How MapReduce shuffle takes advantage of NM's Auxiliary-services
The Shuffle functionality required to run a MapReduce (MR) application is implemented as an
Auxiliary Service. This service starts up a Netty Web Server, and knows how to handle MR specific
shuffle requests from Reduce tasks. The MR AM specifies the service id for the shuffle service,
along with security tokens that may be required. The NM provides the AM with the port on which
the shuffle service is running which is passed onto the Reduce tasks."
How does the AM get the service ID and the port?

View raw message