[ https://issues.apache.org/jira/browse/YARN-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16416582#comment-16416582
]
Eric Yang commented on YARN-7935:
---------------------------------
[~suma.shivaprasad] Docker embedded DNS will use /etc/resolv.conf from host, and filter out
local IP addresses (127.0.0.1 etc), if no entires are available, it will route to 8.8.8.8.
If RegistryDNS is listed in host machine /etc/resolv.conf, then it will be used. There is
no need to manually redefine network alias or CONTAINER_HOSTNAME for spark workload to run.
I don't think we can modify Hadoop code to support docker's user-defined network in 2018
because docker heavily use of NAT to accomplish user defined network. This doesn't work today
with Hadoop because Hadoop only understand 1 dimensional network due to constrain check of
client and server/IP mapping for preventing replay attack. Hence, the proposed feature of
use --link=CONTAINER_NAME:ALIAS for docker embedded DNS might not work as well as you have
hoped for Hadoop 3.x.
> Expose container's hostname to applications running within the docker container
> -------------------------------------------------------------------------------
>
> Key: YARN-7935
> URL: https://issues.apache.org/jira/browse/YARN-7935
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
> Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
> Priority: Major
> Attachments: YARN-7935.1.patch, YARN-7935.2.patch, YARN-7935.3.patch
>
>
> Some applications have a need to bind to the container's hostname (like Spark) which
is different from the NodeManager's hostname(NM_HOST which is available as an env during container
launch) when launched through Docker runtime. The container's hostname can be exposed to
applications via an env CONTAINER_HOSTNAME. Another potential candidate is the container's
IP but this can be addressed in a separate jira.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org
|