hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jian He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4757) [Umbrella] Simplified discovery of services via DNS mechanisms
Date Mon, 06 Jun 2016 21:39:21 GMT

    [ https://issues.apache.org/jira/browse/YARN-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317309#comment-15317309
] 

Jian He commented on YARN-4757:
-------------------------------

[~jmaron], thanks for your reply ! few other questions: 
bq. The idea here was to support a use case in which the YARN DNS server was designated as
a resolver for a suite of hosts in the cluster.
Do you mean this flag will be used to enable/disable dns functionality if the DNS server is
hosted in RM ?
bq. In those instances, queries that it itself could not resolve would have to be forwarded
to a "primary" DNS server for resolution 
 I dont quite know what the SimpleResolver can do. Does it behave like a normal DNS server
which can answer non-YARN queries ?  I thought the flow is that if the primary server cannot
answer the query, that will be forwarded to yarn dns. Not that yarn dns forward to the primary
server.
bq.  I assume the reason is because the ZK path to the node relates the user, application,
and component name.
After closer look, IIUC, this patch assumes the last entry of the zk path is container Id
only.  If it is component name, then the BaseServiceRecordProcessor#getContainerIDName will
break. 

> [Umbrella] Simplified discovery of services via DNS mechanisms
> --------------------------------------------------------------
>
>                 Key: YARN-4757
>                 URL: https://issues.apache.org/jira/browse/YARN-4757
>             Project: Hadoop YARN
>          Issue Type: New Feature
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Jonathan Maron
>         Attachments: 0001-YARN-4757-Initial-code-submission-for-DNS-Service.patch, YARN-4757-
Simplified discovery of services via DNS mechanisms.pdf, YARN-4757-YARN-4757.001.patch, YARN-4757-YARN-4757.002.patch,
YARN-4757-YARN-4757.003.patch
>
>
> [See overview doc at YARN-4692, copying the sub-section (3.2.10.2) to track all related
efforts.]
> In addition to completing the present story of service­-registry (YARN-913), we also
need to simplify the access to the registry entries. The existing read mechanisms of the YARN
Service Registry are currently limited to a registry specific (java) API and a REST interface.
In practice, this makes it very difficult for wiring up existing clients and services. For
e.g, dynamic configuration of dependent end­points of a service is not easy to implement
using the present registry­-read mechanisms, *without* code-changes to existing services.
> A good solution to this is to expose the registry information through a more generic
and widely used discovery mechanism: DNS. Service Discovery via DNS uses the well-­known
DNS interfaces to browse the network for services. YARN-913 in fact talked about such a DNS
based mechanism but left it as a future task. (Task) Having the registry information exposed
via DNS simplifies the life of services.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message