Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7B17CF3ED for ; Tue, 28 May 2013 22:43:50 +0000 (UTC) Received: (qmail 37475 invoked by uid 500); 28 May 2013 22:43:45 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 37165 invoked by uid 500); 28 May 2013 22:43:45 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 37158 invoked by uid 99); 28 May 2013 22:43:45 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 May 2013 22:43:45 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of john.lilley@redpoint.net designates 206.225.164.217 as permitted sender) Received: from [206.225.164.217] (HELO hub021-nj-2.exch021.serverdata.net) (206.225.164.217) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 May 2013 22:43:37 +0000 Received: from MBX021-E3-NJ-2.exch021.domain.local ([10.240.4.78]) by HUB021-NJ-2.exch021.domain.local ([10.240.4.33]) with mapi id 14.02.0318.001; Tue, 28 May 2013 15:43:16 -0700 From: John Lilley To: "user@hadoop.apache.org" Subject: NM/AM interaction Thread-Topic: NM/AM interaction Thread-Index: Ac5b9JkW+oyEux6DR7ebrfMM8rN/Uw== Date: Tue, 28 May 2013 22:43:16 +0000 Message-ID: <869970D71E26D7498BDAC4E1CA92226B658A4528@MBX021-E3-NJ-2.exch021.domain.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [173.160.43.61] Content-Type: multipart/alternative; boundary="_000_869970D71E26D7498BDAC4E1CA92226B658A4528MBX021E3NJ2exch_" MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org --_000_869970D71E26D7498BDAC4E1CA92226B658A4528MBX021E3NJ2exch_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable I was reading from the HortonWorks blog: "How MapReduce shuffle takes advantage of NM's Auxiliary-services The Shuffle functionality required to run a MapReduce (MR) application is i= mplemented as an Auxiliary Service. This service starts up a Netty Web Serv= er, and knows how to handle MR specific shuffle requests from Reduce tasks.= The MR AM specifies the service id for the shuffle service, along with sec= urity tokens that may be required. The NM provides the AM with the port on = which the shuffle service is running which is passed onto the Reduce tasks.= " How does the AM get the service ID and the port? Thanks, John --_000_869970D71E26D7498BDAC4E1CA92226B658A4528MBX021E3NJ2exch_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

I was reading from the HortonWorks blog:<= /p>

“How MapReduce shuffle takes advantage of NM&#= 8217;s Auxiliary-services

The Shuffle functionality required to run a MapReduc= e (MR) application is implemented as an Auxiliary Service. This service sta= rts up a Netty Web Server, and knows how to handle MR specific shuffle requ= ests from Reduce tasks. The MR AM specifies the service id for the shuffle service, along with security toke= ns that may be required. The NM provides the AM with the port on which the = shuffle service is running which is passed onto the Reduce tasks.”

How does the AM get the service ID and the port?

Thanks,

John

 

--_000_869970D71E26D7498BDAC4E1CA92226B658A4528MBX021E3NJ2exch_--