hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Maron (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4757) [Umbrella] Simplified discovery of services via DNS mechanisms
Date Mon, 06 Jun 2016 22:12:21 GMT

    [ https://issues.apache.org/jira/browse/YARN-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317372#comment-15317372

Jonathan Maron commented on YARN-4757:

Do you mean this flag will be used to enable/disable dns functionality if the DNS server is
hosted in RM ?

Oh - good point.  At one point I was looking at embedding the server, but at this point that
is not the case so the flag is probably unnecessary.  I'll remove it.

 dont quite know what the SimpleResolver can do. Does it behave like a normal DNS server
which can answer non-YARN queries ? I thought the flow is that if the primary server cannot
answer the query, that will be forwarded to yarn dns. Not that yarn dns forward to the primary

The SimpleResolver acts as a DNS client in this instance.  I was exploring the idea of allowing
the YARN DNS server to server as a "primary" server by indirectly supporting record retrieval
for records outside of the YARN zone.  I now feel that is probably very unlikely, so I can
remove that feature.

After closer look, IIUC, this patch assumes the last entry of the zk path is container Id
only. If it is component name, then the BaseServiceRecordProcessor#getContainerIDName will

It is container ID in practice, which may actually conflict with the YARN Registry documentation.
 I may have mis-spoken when I indicated the component name is related in the path.  So my
belief is that the correct and is correlated to the real work implementation.

> [Umbrella] Simplified discovery of services via DNS mechanisms
> --------------------------------------------------------------
>                 Key: YARN-4757
>                 URL: https://issues.apache.org/jira/browse/YARN-4757
>             Project: Hadoop YARN
>          Issue Type: New Feature
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Jonathan Maron
>         Attachments: 0001-YARN-4757-Initial-code-submission-for-DNS-Service.patch, YARN-4757-
Simplified discovery of services via DNS mechanisms.pdf, YARN-4757-YARN-4757.001.patch, YARN-4757-YARN-4757.002.patch,
> [See overview doc at YARN-4692, copying the sub-section ( to track all related
> In addition to completing the present story of service­-registry (YARN-913), we also
need to simplify the access to the registry entries. The existing read mechanisms of the YARN
Service Registry are currently limited to a registry specific (java) API and a REST interface.
In practice, this makes it very difficult for wiring up existing clients and services. For
e.g, dynamic configuration of dependent end­points of a service is not easy to implement
using the present registry­-read mechanisms, *without* code-changes to existing services.
> A good solution to this is to expose the registry information through a more generic
and widely used discovery mechanism: DNS. Service Discovery via DNS uses the well-­known
DNS interfaces to browse the network for services. YARN-913 in fact talked about such a DNS
based mechanism but left it as a future task. (Task) Having the registry information exposed
via DNS simplifies the life of services.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message