hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shane Kumpf (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7797) Docker host network can not obtain IP address for RegistryDNS
Date Wed, 24 Jan 2018 14:21:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16337644#comment-16337644

Shane Kumpf commented on YARN-7797:

Thanks for the patch, [~eyang]! This will be a great feature for testing.
{quote}Does docker inspect always return the IP address right away?
In my testing, there is a delay. I think we need to also check the container's env and ensure
that \{{YARN_CONTAINER_RUNTIME_DOCKER_CONTAINER_NETWORK}} is set to host before we invoke
the logic to set the container IP/host to the match the NM.
{quote}My second thought is that if the IP is empty but the hostname is populated, perhaps
we should return the host address for the IP and the docker inspect hostname for the hostname
I believe this is the right approach, but there are caveats to this. If used with {{RegistryDNS}},
we will have multiple hostnames returned in the PTR lookup, which can lead to non-deterministic
behavior. If the NM is restarted, and the NM host is using {{RegistryDNS}} as its resolver,
the NM may report in to the RM with our container hostname. Also, if no hostname is supplied,
I've regularly seen _localhost_ and _localhost.localdomain_ as the container hostname, and
I don't think creating entries in {{RegistryDNS}} would be appropriate for those hostnames.
Perhaps it would be good to return the NM hostname in that case? Despite the caveats, using
the container hostname allows us to validate DNS and forward lookups will work for containers,
which is very valuable in testing, so I do think we should use the container hostname where

Given the potential issues, perhaps it would be good to add a small warning to the docs that _--net=host_ is
really only appropriate for testing and/or administrators that understand the implications
of enabling it?

> Docker host network can not obtain IP address for RegistryDNS
> -------------------------------------------------------------
>                 Key: YARN-7797
>                 URL: https://issues.apache.org/jira/browse/YARN-7797
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager
>    Affects Versions: 3.1.0
>            Reporter: Eric Yang
>            Assignee: Eric Yang
>            Priority: Major
>         Attachments: YARN-7797.001.patch
> When docker is configured to use host network, docker inspect command does not return
IP address of the container.  This prevents IP information to be collected for RegistryDNS
to register a hostname entry for the docker container.
> The proposed solution is to intelligently detect the docker network deployment method,
and report back host IP address for RegistryDNS.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message