hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7935) Expose container's hostname to applications running within the docker container
Date Wed, 28 Mar 2018 01:53:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16416582#comment-16416582

Eric Yang commented on YARN-7935:

[~suma.shivaprasad] Docker embedded DNS will use /etc/resolv.conf from host, and filter out
local IP addresses ( etc), if no entires are available, it will route to
 If RegistryDNS is listed in host machine /etc/resolv.conf, then it will be used.  There is
no need to manually redefine network alias or CONTAINER_HOSTNAME for spark workload to run.
 I don't think we can modify Hadoop code to support docker's user-defined network in 2018
because docker heavily use of NAT to accomplish user defined network.  This doesn't work today
with Hadoop because Hadoop only understand 1 dimensional network due to constrain check of
client and server/IP mapping for preventing replay attack.  Hence, the proposed feature of
use --link=CONTAINER_NAME:ALIAS for docker embedded DNS  might not work as well as you have
hoped for Hadoop 3.x.

> Expose container's hostname to applications running within the docker container
> -------------------------------------------------------------------------------
>                 Key: YARN-7935
>                 URL: https://issues.apache.org/jira/browse/YARN-7935
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: yarn
>            Reporter: Suma Shivaprasad
>            Assignee: Suma Shivaprasad
>            Priority: Major
>         Attachments: YARN-7935.1.patch, YARN-7935.2.patch, YARN-7935.3.patch
> Some applications have a need to bind to the container's hostname (like Spark) which
is different from the NodeManager's hostname(NM_HOST which is available as an env during container
launch) when launched through Docker runtime. The container's hostname can be exposed to
applications via an env CONTAINER_HOSTNAME. Another potential candidate is the container's
IP but this can be addressed in a separate jira.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message