Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Tue, 5 Apr 2016 14:48:25 +0000 (UTC)
From: "Allen Wittenauer (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12946681.1457039063000.138998.1459867705781@Atlassian.JIRA>
In-Reply-To: <JIRA.12946681.1457039063000@Atlassian.JIRA>
References: <JIRA.12946681.1457039063000@Atlassian.JIRA>
 <JIRA.12946681.1457039063332@arcas>
Subject: [jira] [Commented] (YARN-4757) [Umbrella] Simplified discovery of
 services via DNS mechanisms
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/YARN-4757?page=3Dcom.atlassian.=
jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D15226=
395#comment-15226395 ]=20

Allen Wittenauer commented on YARN-4757:
----------------------------------------

bq. May I ask why?

In the vast majority of cases I've seen, there's been primarily two reasons=
:

* performance.  A specialty, high-end DNS server (e.g., Infoblox, PowerDNS,=
 etc) will typically blow the doors off of home grown solutions... esp if J=
ava GC pauses are involved. ;)  Never mind that having forced, local resolu=
tion is a HUGE win when dealing with heavy loads or high latency connection=
s.  Why should I blow out my local DNS cache when I can just AXFR it to beg=
in with?  Latency in DNS queries matters; it's one of the reasons why serve=
rs at places like Y! ran (and probably still do) with full grown DNS caches=
 using BIND on the local nodes.

* limiting exposure.  It's a much simpler network and security monitoring d=
esign if the DNS pathways are limited.  If I know that DNS server X can aut=
horitatively answer questions for my entire corporate network, then I know =
that the only DNS traffic I should be seeing are zone transfers, especially=
 if I also close off access to the Internet (think data center).  If I see =
any extra traffic, something bad is likely going on.

bq.  If a zone key is created and properly configured, isn't the authentici=
ty of the records assured?=20

As you can see, that's mostly an irrelevant question.

You're also assuming that group A trusts group B. The folks who run the cor=
porate DNS are not the same people who run the data center DNS who are not =
the same people that run the Hadoop systems who are not the same people who=
 are actually using the Hadoop systems as end users.  ANY sort of insurance=
 policy (in this case, AFXR with bonus points for IXFR) enable a lot more t=
rust.

bq. Do we need to require the support of secure zone transfers?

Absolutely.  It's trivial to run out of fingers of big name companies will =
dismiss this totally if they can't do resolution from their own servers.


> [Umbrella] Simplified discovery of services via DNS mechanisms
> --------------------------------------------------------------
>
>                 Key: YARN-4757
>                 URL: https://issues.apache.org/jira/browse/YARN-4757
>             Project: Hadoop YARN
>          Issue Type: New Feature
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Jonathan Maron
>         Attachments: YARN-4757- Simplified discovery of services via DNS =
mechanisms.pdf
>
>
> [See overview doc at YARN-4692, copying the sub-section (3.2.10.2) to tra=
ck all related efforts.]
> In addition to completing the present story of service=C2=AD-registry (YA=
RN-913), we also need to simplify the access to the registry entries. The e=
xisting read mechanisms of the YARN Service Registry are currently limited =
to a registry specific (java) API and a REST interface. In practice, this m=
akes it very difficult for wiring up existing clients and services. For e.g=
, dynamic configuration of dependent end=C2=ADpoints of a service is not ea=
sy to implement using the present registry=C2=AD-read mechanisms, *without*=
 code-changes to existing services.
> A good solution to this is to expose the registry information through a m=
ore generic and widely used discovery mechanism: DNS. Service Discovery via=
 DNS uses the well-=C2=ADknown DNS interfaces to browse the network for ser=
vices. YARN-913 in fact talked about such a DNS based mechanism but left it=
 as a future task. (Task) Having the registry information exposed via DNS s=
implifies the life of services.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)