hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4757) [Umbrella] Simplified discovery of services via DNS mechanisms
Date Wed, 23 Mar 2016 18:28:25 GMT

    [ https://issues.apache.org/jira/browse/YARN-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208925#comment-15208925

Allen Wittenauer commented on YARN-4757:

I did a quick pass, so I'll need to read more in-depth, but I have some concerns:

a) I'm still not sure what value registering A records are here when you can point a SRV record
in the fake DNS zone to an existing host in an existing zone using the existing DNS services.
 This eliminates a ton of corner cases (split zones, NAT, multi-nics, etc) that will need
to be covered when registering As.

b) The BIND cons are very... odd:
* I'm not particularly sure what you find complex about BIND?  Most named.conf's aren't complex
and rarely change after initial install in my experience.  Managing the zone files isn't particularly
hard and lots of tools exist in this space for large scale deployments.
* You're effectively trading multiple instances of BIND for multiple instances of ZK.
* I don't understand the 'stateful' complaint given that, again, you're trading state of BIND
for the state stored in ZK.
* Better security requirements sounds like a good thing to me...

c) Where are the comparisons with other open source DNS solutions?  Doesn't Manta already
have something exactly like this already?

d) The NIH DNS server solution:
* "No operational dependencies on elements external to the Hadoop cluster"... Nothing says
"thrown over the fence" like "no operational dependencies" when stated by a developer.
* it's unknown how well it's going to perform at scale.
* no idea how secure it's actually going to be--spoofing, MITM, etc.
* admins have zero experience with it vs. pre-existing solutions so will be a knowledge gap.
 (Never mind the "the software doesn't exist yet so how can someone have experience with it?"
* increases the source footprint for what is effectively a solved problem

> [Umbrella] Simplified discovery of services via DNS mechanisms
> --------------------------------------------------------------
>                 Key: YARN-4757
>                 URL: https://issues.apache.org/jira/browse/YARN-4757
>             Project: Hadoop YARN
>          Issue Type: New Feature
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Jonathan Maron
>         Attachments: YARN-4757- Simplified discovery of services via DNS mechanisms.pdf
> [See overview doc at YARN-4692, copying the sub-section ( to track all related
> In addition to completing the present story of service­-registry (YARN-913), we also
need to simplify the access to the registry entries. The existing read mechanisms of the YARN
Service Registry are currently limited to a registry specific (java) API and a REST interface.
In practice, this makes it very difficult for wiring up existing clients and services. For
e.g, dynamic configuration of dependent end­points of a service is not easy to implement
using the present registry­-read mechanisms, *without* code-changes to existing services.
> A good solution to this is to expose the registry information through a more generic
and widely used discovery mechanism: DNS. Service Discovery via DNS uses the well-­known
DNS interfaces to browse the network for services. YARN-913 in fact talked about such a DNS
based mechanism but left it as a future task. (Task) Having the registry information exposed
via DNS simplifies the life of services.

This message was sent by Atlassian JIRA

View raw message