Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 55803184FF for ; Tue, 8 Dec 2015 16:38:11 +0000 (UTC) Received: (qmail 68613 invoked by uid 500); 8 Dec 2015 16:38:00 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 68508 invoked by uid 500); 8 Dec 2015 16:38:00 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 68494 invoked by uid 99); 8 Dec 2015 16:38:00 -0000 Received: from Unknown (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 08 Dec 2015 16:38:00 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id F08F31A35F6 for ; Tue, 8 Dec 2015 16:37:59 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 4.902 X-Spam-Level: **** X-Spam-Status: No, score=4.902 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, KAM_BADIPHTTP=2, NORMAL_HTTP_TO_IP=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, WEIRD_PORT=0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-eu-west.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id QPPS6yn4K8UE for ; Tue, 8 Dec 2015 16:37:50 +0000 (UTC) Received: from mail-ig0-f171.google.com (mail-ig0-f171.google.com [209.85.213.171]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with ESMTPS id 1F41D20C70 for ; Tue, 8 Dec 2015 16:37:49 +0000 (UTC) Received: by igcmv3 with SMTP id mv3so104855713igc.0 for ; Tue, 08 Dec 2015 08:32:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=TxT4mOh4D425C6bZsuXIEIAyAyT3+fAjZA5D/84j3rw=; b=LcIac10ugWpPr7l0t3JuFNGEYu24hdKMILlxmZrfbQUa3wd+Nk17wherB63rPpceRu 4ssXSs43RDVApkXvhml1FIvuyubFGyPLTwKauBRC34ZnpW08gxIKy/zsgkObozaD8Yhx C1Dk6oC4Ld9A26Op9EL6bOxdB/chU5pQ9XBL9/51faE2fMUJiz1hLCg6ItES6TqBAmz/ hhF3uyzf881p5Bvs+3LM/fPadMX/6Eg4iJKKPupgaWzdBZIJ0sNTmMkab3x2YJE9g8Nq pK42pACb+rmAfQ7fUCmjue5/bJqJ+vdFr4cni28KKX2ZW9Uzgsi5xtahBr5KKbieeHcl L8Tw== MIME-Version: 1.0 X-Received: by 10.51.17.34 with SMTP id gb2mr21875297igd.45.1449592367074; Tue, 08 Dec 2015 08:32:47 -0800 (PST) Received: by 10.64.1.133 with HTTP; Tue, 8 Dec 2015 08:32:46 -0800 (PST) In-Reply-To: References: Date: Tue, 8 Dec 2015 22:02:46 +0530 Message-ID: Subject: Re: nodemanager listen on 0.0.0.0 From: Varun Saxena To: Harsh J Cc: Varun Vasudev , Li Li , user@hadoop.apache.org Content-Type: multipart/alternative; boundary=001a1135ed321bd35d05266586aa --001a1135ed321bd35d05266586aa Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable There is already a JIRA for it - MAPREDUCE-5938(and other related JIRAs' MAPREDUCE-6402 and YARN-4119) Regards, Varun Saxena On Tue, Dec 8, 2015 at 9:56 PM, Harsh J wrote: > Hello, > > Could you file a JIRA for this please? Currently the ShuffleHandler will > always bind to wildcard address due to the code being that way (in both > branch-2 and trunk presently: > https://github.com/apache/hadoop/blob/release-2.7.1/hadoop-mapreduce-proj= ect/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/o= rg/apache/hadoop/mapred/ShuffleHandler.java#L388 > ). > > Your current workaround would be to use a firewall to drop out external > connections to that port. > > On Tue, Dec 8, 2015 at 9:30 PM Varun Vasudev wrote: > >> Hi Li, >> >> What have you set yarn.nodemanager.bind-host to? From >> https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-common/yar= n-default.xml >> - >> >> "The actual address the server will bind to. If this optional address is >> set, the RPC and webapp servers will bind to this address and the >> port specified in >> yarn.nodemanager.address and yarn.nodemanager.webapp.address, >> respectively. This is >> most useful for making NM listen to all interfaces by setting to >> 0.0.0.0=E2=80=9D >> >> -Varun >> >> >> >> On 12/8/15, 2:57 PM, "Li Li" wrote: >> >> > I want to setup a yarn cluster. It's not allowed to visit from >> >public network so I changed many configuration files to let all >> >service bind to the internal network address. >> > All thing is fine exception the nodemanager which bind >> >0.0.0.0:13562. I searched the default-mapred-default.xml and found it >> >mapreduce.shuffle.port. >> > Which configuration file should I change to let nodemanager listen >> >on internal address instead of 0.0.0.0? >> > >> >--------------------------------------------------------------------- >> >To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org >> >For additional commands, e-mail: user-help@hadoop.apache.org >> > >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org >> For additional commands, e-mail: user-help@hadoop.apache.org >> >> --001a1135ed321bd35d05266586aa Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
There is already a JIRA for it - MAPREDUCE-5938(and other = related JIRAs' MAPREDUCE-6402 and YARN-4119)

Regards= ,
Varun Saxena

On Tue, Dec 8, 2015 at 9:56 PM, Harsh J <harsh@clouder= a.com> wrote:
Hello,

Could you file a JIRA for this please? Curren= tly the ShuffleHandler will always bind to wildcard address due to the code= being that way (in both branch-2 and trunk presently:=C2=A0https://githu= b.com/apache/hadoop/blob/release-2.7.1/hadoop-mapreduce-project/hadoop-mapr= educe-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoo= p/mapred/ShuffleHandler.java#L388).

Your curre= nt workaround would be to use a firewall to drop out external connections t= o that port.

On Tue, Dec 8, 2015 at 9:30 PM Varun V= asudev <vvasude= v@apache.org> wrote:
Hi Li,<= br>
What have you set yarn.nodemanager.bind-host to? From https://hadoop.apache.org/docs/r2.7.= 1/hadoop-yarn/hadoop-yarn-common/yarn-default.xml -

"The actual address the server will bind to. If this optional address = is
=C2=A0 =C2=A0 =C2=A0 set, the RPC and webapp servers will bind to this addr= ess and the port specified in
=C2=A0 =C2=A0 =C2=A0 yarn.nodemanager.address and yarn.nodemanager.webapp.a= ddress, respectively. This is
=C2=A0 =C2=A0 =C2=A0 most useful for making NM listen to all interfaces by = setting to 0.0.0.0=E2=80=9D

-Varun



On 12/8/15, 2:57 PM, "Li Li" <fancyerii@gmail.com> wrote:

>=C2=A0 =C2=A0I want to setup a yarn cluster. It's not allowed to vi= sit from
>public network so I changed many configuration files to let all
>service bind to the internal network address.
>=C2=A0 =C2=A0All thing is fine exception the nodemanager which bind
>0= .0.0.0:13562. I searched the default-mapred-default.xml and found it >mapreduce.shuffle.port.
>=C2=A0 =C2=A0Which configuration file should I change to let nodemanage= r listen
>on internal address instead of 0.0.0.0?
>
>--------------------------------------------------------------------- >To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
>For additional commands, e-mail: user-help@hadoop.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org


--001a1135ed321bd35d05266586aa--