hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suma Shivaprasad (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7935) Expose container's hostname to applications running within the docker container
Date Mon, 26 Mar 2018 18:17:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16414269#comment-16414269

Suma Shivaprasad commented on YARN-7935:

Spark drivers/AM/executor on docker do not go through Registry DNS and uses docker embedded
DNS(in user defined networks) for resolution in the current implementation since spark has
a custom AM and is not a Yarn service. 

Had explored using HOSTNAME/`hostname` for setting the container's hostname but decided to
use a per-configured value instead due to possible injection attacks with /etc/hosts - [https://bugs.launchpad.net/ubuntu/+source/bash/+bug/1507025|https://www.google.com/url?q=https://bugs.launchpad.net/ubuntu/%2Bsource/bash/%2Bbug/1507025&sa=D&ust=1519936820096000&usg=AFQjCNHA75LBYmOwM5b5vTwReS26dWL5Mg]
which could be problematic in 'host' network mode. The security vulnerability seems to be
fixed only in bash versions > 4.4 and may be different in other linux distributions.

Also, the impact of setting hostname through --host in "host" mode seems to differ in behaviour
across docker versions , OS(on MAC,  --host's set value in 'host' networking mode is returned
by `hostname` or $HOSTNAME which is not routable and this differs from linux) and hence --hostname's
value seems inherently unstable to depend on.

> Expose container's hostname to applications running within the docker container
> -------------------------------------------------------------------------------
>                 Key: YARN-7935
>                 URL: https://issues.apache.org/jira/browse/YARN-7935
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: yarn
>            Reporter: Suma Shivaprasad
>            Assignee: Suma Shivaprasad
>            Priority: Major
>         Attachments: YARN-7935.1.patch, YARN-7935.2.patch, YARN-7935.3.patch
> Some applications have a need to bind to the container's hostname (like Spark) which
is different from the NodeManager's hostname(NM_HOST which is available as an env during container
launch) when launched through Docker runtime. The container's hostname can be exposed to
applications via an env CONTAINER_HOSTNAME. Another potential candidate is the container's
IP but this can be addressed in a separate jira.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message