hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Maron (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4757) [Umbrella] Simplified discovery of services via DNS mechanisms
Date Mon, 06 Jun 2016 22:12:21 GMT

    [ https://issues.apache.org/jira/browse/YARN-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317372#comment-15317372
] 

Jonathan Maron commented on YARN-4757:
--------------------------------------

{quote}
Do you mean this flag will be used to enable/disable dns functionality if the DNS server is
hosted in RM ?
{quote}

Oh - good point.  At one point I was looking at embedding the server, but at this point that
is not the case so the flag is probably unnecessary.  I'll remove it.

{quote}
 dont quite know what the SimpleResolver can do. Does it behave like a normal DNS server
which can answer non-YARN queries ? I thought the flow is that if the primary server cannot
answer the query, that will be forwarded to yarn dns. Not that yarn dns forward to the primary
server.
{quote}

The SimpleResolver acts as a DNS client in this instance.  I was exploring the idea of allowing
the YARN DNS server to server as a "primary" server by indirectly supporting record retrieval
for records outside of the YARN zone.  I now feel that is probably very unlikely, so I can
remove that feature.

{quote}
After closer look, IIUC, this patch assumes the last entry of the zk path is container Id
only. If it is component name, then the BaseServiceRecordProcessor#getContainerIDName will
break.
{quote}

It is container ID in practice, which may actually conflict with the YARN Registry documentation.
 I may have mis-spoken when I indicated the component name is related in the path.  So my
belief is that the correct and is correlated to the real work implementation.

> [Umbrella] Simplified discovery of services via DNS mechanisms
> --------------------------------------------------------------
>
>                 Key: YARN-4757
>                 URL: https://issues.apache.org/jira/browse/YARN-4757
>             Project: Hadoop YARN
>          Issue Type: New Feature
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Jonathan Maron
>         Attachments: 0001-YARN-4757-Initial-code-submission-for-DNS-Service.patch, YARN-4757-
Simplified discovery of services via DNS mechanisms.pdf, YARN-4757-YARN-4757.001.patch, YARN-4757-YARN-4757.002.patch,
YARN-4757-YARN-4757.003.patch
>
>
> [See overview doc at YARN-4692, copying the sub-section (3.2.10.2) to track all related
efforts.]
> In addition to completing the present story of service­-registry (YARN-913), we also
need to simplify the access to the registry entries. The existing read mechanisms of the YARN
Service Registry are currently limited to a registry specific (java) API and a REST interface.
In practice, this makes it very difficult for wiring up existing clients and services. For
e.g, dynamic configuration of dependent end­points of a service is not easy to implement
using the present registry­-read mechanisms, *without* code-changes to existing services.
> A good solution to this is to expose the registry information through a more generic
and widely used discovery mechanism: DNS. Service Discovery via DNS uses the well-­known
DNS interfaces to browse the network for services. YARN-913 in fact talked about such a DNS
based mechanism but left it as a future task. (Task) Having the registry information exposed
via DNS simplifies the life of services.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message