Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
MIME-Version: 1.0
Date: Fri, 20 Nov 2015 15:50:35 +0100
Message-ID: 
 <CAFwvhUnTpG1_b3vaR4CoKFUq_uy5mMME8mP0-vJUyiiM9pDsZw@mail.gmail.com>
Subject: Yarn: how to specify the hostname/IP for communication between the
 ResourceManager and the NodeManager for a Kubernetes deployment.
From: geoffroy chollon <un.moins.1.egal.0@gmail.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=001a11444efa84d5710524f9ff02

--001a11444efa84d5710524f9ff02
Content-Type: text/plain; charset=UTF-8

Hello,


Tl;dr question: How to make the NodeManager give the
"yarn.nodemanager.hostname" value to the Yarn ResourceManager during
registration for IPC communication.


Explanations:

I have trouble making a Yarn cluster works in a Kubernetes deployment where
every components run as a Docker container.

When the NodeManager registers with the Yarn ResourceManager, it give its
hostname for NodeId. Then the ResourceManager uses this NodeId to
communicate with the node.

The problem is that the hostname comes from system calls and not from the
Yarn's configuration files. The ResourceManager is unable to talk to the
NodeManager because the container hostname is invalid in the DNS context.
Moreover, Docker disallows a dynamic change of a container's hostname.

My Yarn NodeManager's configuration looks like this:

*      <property>
*        <name>yarn.resourcemanager.hostname</name>
*        <value>yarnrm-int-svc.default.svc.cluster.docker</value>
*      </property>
*
*      <property>
*        <name>yarn.nodemanager.hostname</name>
*        <value>100-66-10-4.default.pod.cluster.docker</value>
*      </property>
*      <property>
*        <name>yarn.nodemanager.address</name>
*        <value>100-66-10-4.default.pod.cluster.docker:8111</value>
*      </property>
*      <property>
*        <name>yarn.nodemanager.webapp.address</name>
*        <value>100-66-10-4.default.pod.cluster.docker:8042</value>
*      </property>
*      <property>
*        <name>yarn.nodemanager.localizer.address</name>
*        <value>100-66-10-4.default.pod.cluster.docker:8040</value>
*      </property>

I tried to simply use IP but it didn't work either. For now I had to fork
the Kubernetes's DNS plugin to please Yarn but its not a proper solution.

It's difficult to understand why the NodeId not configurable in this setup.
If so, why do the "yarn.nodemanager.hostname" variable exists in the first
place ?.
Anyway, if someone knows a solution to this problem it would be very
helpful.


Thanks
Geoffroy

--001a11444efa84d5710524f9ff02
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Hello,<br></div><div><br></div><div><br></div><div>Tl=
;dr question: How to make the NodeManager give the &quot;yarn.nodemanager.h=
ostname&quot; value to the Yarn ResourceManager during registration for IPC=
 communication.</div><div><br></div><div><br></div><div>Explanations:</div>=
<div><br></div><div>I have trouble making a Yarn cluster works in a Kuberne=
tes deployment where every components run as a Docker container.</div><div>=
<br></div><div>When the NodeManager registers with the Yarn ResourceManager=
, it give its hostname for NodeId. Then the ResourceManager uses this NodeI=
d to communicate with the node.</div><div><br></div><div>The problem is tha=
t the hostname comes from system calls and not from the Yarn&#39;s configur=
ation files. The ResourceManager is unable to talk to the NodeManager becau=
se the container hostname is invalid in the DNS context. Moreover, Docker d=
isallows a dynamic change of a container&#39;s hostname.=C2=A0</div><div><b=
r></div><div>My Yarn NodeManager&#39;s configuration looks like this:</div>=
<div><br></div><div>* =C2=A0 =C2=A0 =C2=A0&lt;property&gt;</div><div>* =C2=
=A0 =C2=A0 =C2=A0 =C2=A0&lt;name&gt;yarn.resourcemanager.hostname&lt;/name&=
gt;</div><div>* =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;value&gt;yarnrm-int-svc.defa=
ult.svc.cluster.docker&lt;/value&gt;</div><div>* =C2=A0 =C2=A0 =C2=A0&lt;/p=
roperty&gt;</div><div>*</div><div>* =C2=A0 =C2=A0 =C2=A0&lt;property&gt;</d=
iv><div>* =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;name&gt;yarn.nodemanager.hostname&=
lt;/name&gt;</div><div>* =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;value&gt;100-66-10-=
4.default.pod.cluster.docker&lt;/value&gt;</div><div>* =C2=A0 =C2=A0 =C2=A0=
&lt;/property&gt;</div><div>* =C2=A0 =C2=A0 =C2=A0&lt;property&gt;</div><di=
v>* =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;name&gt;yarn.nodemanager.address&lt;/nam=
e&gt;</div><div>* =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;value&gt;100-66-10-4.defau=
lt.pod.cluster.docker:8111&lt;/value&gt;</div><div>* =C2=A0 =C2=A0 =C2=A0&l=
t;/property&gt;</div><div>* =C2=A0 =C2=A0 =C2=A0&lt;property&gt;</div><div>=
* =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;name&gt;yarn.nodemanager.webapp.address&lt=
;/name&gt;</div><div>* =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;value&gt;100-66-10-4.=
default.pod.cluster.docker:8042&lt;/value&gt;</div><div>* =C2=A0 =C2=A0 =C2=
=A0&lt;/property&gt;</div><div>* =C2=A0 =C2=A0 =C2=A0&lt;property&gt;</div>=
<div>* =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;name&gt;yarn.nodemanager.localizer.ad=
dress&lt;/name&gt;</div><div>* =C2=A0 =C2=A0 =C2=A0 =C2=A0&lt;value&gt;100-=
66-10-4.default.pod.cluster.docker:8040&lt;/value&gt;</div><div>* =C2=A0 =
=C2=A0 =C2=A0&lt;/property&gt;</div><div><br></div><div>I tried to simply u=
se IP but it didn&#39;t work either. For now I had to fork the Kubernetes&#=
39;s DNS plugin to please Yarn but its not a proper solution.</div><div><br=
></div><div>It&#39;s difficult to understand why the NodeId not configurabl=
e in this setup. If so, why do the &quot;yarn.nodemanager.hostname&quot; va=
riable exists in the first place ?.</div><div>Anyway, if someone knows a so=
lution to this problem it would be very helpful.</div><div><br></div><div><=
br></div><div>Thanks</div><div>Geoffroy</div></div>

--001a11444efa84d5710524f9ff02--