hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shane Kumpf (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-7935) Expose container's hostname to applications running within the docker container
Date Wed, 28 Mar 2018 11:11:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417175#comment-16417175
] 

Shane Kumpf edited comment on YARN-7935 at 3/28/18 11:10 AM:
-------------------------------------------------------------

{quote}Docker embedded DNS will use /etc/resolv.conf from host, and filter out local IP addresses
(127.0.0.1 etc), if no entires are available, it will route to 8.8.8.8
{quote}
[~eyang] this isn't true for overlay networks. You can't assume Registry DNS will be in use
and it won't be used by some of these network types without additional modifications to Hadoop
({{--dns}} for {{docker run}}).

{quote}I am concerned that some end user code will end up invoking InetAddress Java class{quote}
This will use the IP of the container and whatever resolver the container is configured to
use. Adding this environment variable doesn't change that.

I'm not seeing the issue with adding an additional environment variable that is set to the
same value as {{\-\-hostname}} if this solves a problem for a class of application. No one
is proposing modifying Hadoop IPC code to support NAT here or to use the {{--link}} feature,
just adding an additional environment variable in non-entrypoint mode. Can you elaborate on
the exact issue you see this new environment variable causing?


was (Author: shanekumpf@gmail.com):
{quote}Docker embedded DNS will use /etc/resolv.conf from host, and filter out local IP addresses
(127.0.0.1 etc), if no entires are available, it will route to 8.8.8.8
{quote}
[~eyang] this isn't true for overlay networks. You can't assume Registry DNS will be in use
and it won't be used by some of these network types without additional modifications to Hadoop
({{--dns}} for {{docker run}}).

{quote}I am concerned that some end user code will end up invoking InetAddress Java class{quote}
This will use the IP of the container and whatever resolver the container is configured to
use. Adding this environment variable doesn't change that.

I'm not seeing the issue with adding an additional environment variable that is set to the
same value as {{--hostname}} if this solves a problem for a class of application. No one is
proposing modifying Hadoop IPC code to support NAT here or to use the {{--link}} feature,
just adding an additional environment variable in non-entrypoint mode. Can you elaborate on
the exact issue you see this new environment variable causing?

> Expose container's hostname to applications running within the docker container
> -------------------------------------------------------------------------------
>
>                 Key: YARN-7935
>                 URL: https://issues.apache.org/jira/browse/YARN-7935
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: yarn
>            Reporter: Suma Shivaprasad
>            Assignee: Suma Shivaprasad
>            Priority: Major
>         Attachments: YARN-7935.1.patch, YARN-7935.2.patch, YARN-7935.3.patch
>
>
> Some applications have a need to bind to the container's hostname (like Spark) which
is different from the NodeManager's hostname(NM_HOST which is available as an env during container
launch) when launched through Docker runtime. The container's hostname can be exposed to
applications via an env CONTAINER_HOSTNAME. Another potential candidate is the container's
IP but this can be addressed in a separate jira.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message