Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 3 Aug 2016 15:45:20 +0000 (UTC)
From: "Varun Vasudev (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12992509.1469519728000.213151.1470239120627@Atlassian.JIRA>
In-Reply-To: <JIRA.12992509.1469519728000@Atlassian.JIRA>
References: <JIRA.12992509.1469519728000@Atlassian.JIRA> <JIRA.12992509.1469519728816@arcas>
Subject: [jira] [Commented] (YARN-5430) Get container's ip and host from NM
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Wed, 03 Aug 2016 15:45:23 -0000


    [ https://issues.apache.org/jira/browse/YARN-5430?page=3Dcom.atlassian.=
jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D15406=
094#comment-15406094 ]=20

Varun Vasudev commented on YARN-5430:
-------------------------------------

Thanks for the patch [~jianhe]!

1)
{code}
+  @Private
+  @Unstable
+  public abstract void setIp(String ip);
+
{code}
We can=E2=80=99t return just one ip. In the non-docker scenario, multiple i=
ps for a host are common and Docker itself has support for multiple ips for=
 a container. We don=E2=80=99t currently support that but it=E2=80=99ll pro=
bably be required for us to support soon. Similarly we should change getIpA=
ndHost and split them up into getIps and getHost calls.

2)
{code}
+  optional string ip =3D 7;
+  optional string host =3D 8;
{code}

Instead of named variables - can we make this a bit more generic and add su=
pport for a simple list of string->list? Then the next time we need to send=
 a new variable, we don=E2=80=99t need to mess around with the protobuf pie=
ces. Something like
{code}
message StringListMapProto {
  required string key =3D 1;
  optional repeated string value =3D 2;
}

and in ContainerStatusProto

{code}
optional repeated StringListMapProto attributes_map =3D 7;
{code}

3)
{code}

+  public String[] getIpAndHost(Container container) {
+    return null;
+  }
{code}

The default implementation shouldn=E2=80=99t return null. We should return =
the host name and ip.

4)
{code}
+  public void setIpAndHost(String[] ipAndHost) {
+    this.ip =3D ipAndHost[0];
+    this.host =3D ipAndHost[1];
+  }
{code}

This may become a problem if the size of the array is more than 2(multiple =
ips)

5)
{code}
+  @Override
+  public String[] getIpAndHost(Container container) {
+    String[] ipAndHost =3D new String[2];
+    try {
+      InetAddress address =3D InetAddress.getLocalHost();
+      ipAndHost[0] =3D address.getHostAddress();
+      ipAndHost[1] =3D address.getHostName();
+    } catch (UnknownHostException e) {
+      LOG.error("Unable to get Local hostname and ip for " + container
+          .getContainerId());
+    }
+    return ipAndHost;
+  }
 }
{code}
Instead of this, we should just use NetUtils.getLocalHostname() and NetUtil=
s.getIPs. we=E2=80=99ll have to modify getIps() because today it expects a =
subnet and we want all ips irrespective of subnet.

6)
{code}
+      LOG.info("Docker inspect output for " + containerId + ": " + output)=
;
{code}
Change log level to debug?

7)
{code}
+    // Be sure to not use space in the argument, otherwise the
+    // extract_values_delim method in container-executor binary
+    // cannot parse the arguments correctly.
+    super.addCommandArguments("--format=3D'{{range.NetworkSettings.Network=
s}}"
+        + "{{.IPAddress}}{{end}},{{.Config.Hostname}}'");
+    super.addCommandArguments(containerName);
+    return this;
{code}

Ugh. This is an ugly one. I thought there needed to be a space between rang=
e and NetworkSettings because the code passed is meant to be a go template.=
 Use this instead - --format=3D'{{range(.NetworkSettings.Networks)}}

8)
{code}
+  if (docker_binary =3D=3D NULL) {
+    docker_binary =3D "docker";
+  }
{code}

Can we make this a common function - the same code is in launch_docker_cont=
ainer_as_user?

> Get container's ip and host from NM
> -----------------------------------
>
>                 Key: YARN-5430
>                 URL: https://issues.apache.org/jira/browse/YARN-5430
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Jian He
>            Assignee: Jian He
>         Attachments: YARN-5430.1.patch, YARN-5430.2.patch, YARN-5430.3.pa=
tch
>
>
> In YARN-4757, we introduced a DNS mechanism for containers. That's based =
on the assumption  that we can get the container's ip and host information =
and store it in the registry-service. This jira aims to get the container's=
 ip and host from the NM, primarily docker container


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org