hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suma Shivaprasad (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7987) Docker container name(--name) needs to be DNS friendly for DNS resolution to work in user defined networks.
Date Fri, 02 Mar 2018 06:06:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383223#comment-16383223
] 

Suma Shivaprasad commented on YARN-7987:
----------------------------------------

Thanks [~shanekumpf@gmail.com] for your inputs. yes you are right - The FQDN is being set
to the <container_name>.<network name> as you mentioned in case of both qualified
and non-qualified container names and not from the domain-name specified in --name or --hostName.
Sorry for not being clear in the previous comment, I meant YARN sets --name to non-qualified
name which fails DNS resolution.

Tested --network-alias with all the available docker network modes and user defined network
types i.e bridge, overlay and macvlan. Here are the observations.
 # network-alias is not a valid option in the "default bridge"and "host" networking modes
and adding this flag fails. Will need to check and add this to docker container runs only
in the case of user defined networks.  Not sure what is the behaviour in case of third party
network plugins. This will need to be validated.
 # --network-alias is not being checked for unique-ness by docker embedded DNS while starting
a container. However --name values are validated to be unique across the network. I could
start two containers with the --network-alias having the same value and the DNS resolution
points to the container's IP which started first. However, since the YARN generated hostnames
(generated from container-id) are unique, this should not be a major issue IMO. 

 

> Docker container name(--name) needs to be DNS friendly for DNS resolution to work in
user defined networks. 
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-7987
>                 URL: https://issues.apache.org/jira/browse/YARN-7987
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Suma Shivaprasad
>            Assignee: Suma Shivaprasad
>            Priority: Major
>
> User defined networks like overlays support DNS resolution through Docker Embedded DNS
which needs the container name (–name parameter value in docker run) to be a FQDN for container
names to be resolved - Please refer documentation [https://docs.docker.com/v17.09/engine/userguide/networking/configure-dns/]
> However Yarn sets the container name to the container's id which is not DNS friendly(eg: container_e26_1519402686002_0035_01_000003)
and is not a FQDN. 
> The proposal is to set a FQDN(eg: ctr-e26-1519402686002-0035-01-000003.domain-name)
as the docker container's name for containers to be able to communicate to each other via
hostnames in user defined networks like overlays, bridges etc. The domain name will be picked
up from the YARN DNS registry configuration (hadoop.registry.dns.domain-name)
>  
>  
>  
>  
>  
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message