hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Joseph Evans (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4757) [Umbrella] Simplified discovery of services via DNS mechanisms
Date Thu, 24 Mar 2016 16:33:25 GMT

    [ https://issues.apache.org/jira/browse/YARN-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210513#comment-15210513

Robert Joseph Evans commented on YARN-4757:

My concern about mutual authentication is mostly around documentation and asking if there
is anything we can do to mitigate possible issues/attacks.  Instead of talking about exact
attacks lets talk about a few accidents that could happen, and can happen today, but are less
likely because when I update my client to use the Registry API I make different assumptions
about things.

Lets say I am running a web service on YARN, and I want my customers to be able to get to
me in through existing tools.  So I set this all up and I have them go to http://api.bobby.yarncluster.myCompany.com:9999/
(or something else that matches the naming convention you had, I don't remember exactly and
it is not relevant)  First of all I have no way to guarantee that 9999 is open on any node,
so it is a pain but I try to launch several web servers, finally get a few to come up, the
others fail and get relaunched on other nodes eventually they are all up and running, and
the AM puts all of them into the registry service.

Things are going well.  Customers are running using my service and everyone is happy.  But
then I do a rolling upgrade and I kill one container and launch a new one on another box.
 In the mean time some other container on the box I was running on grabs port 9999 and brings
up an internal web UI for it.  Now many of my customers trying to hit my web service but get
this other process and see 404 errors, etc.  Because DNS is eventual consistency, and there
is a lot of caching happening, there is a race.  If the client does not authenticate the server,
like with https, then someone malicious could exploit this to do all kinds of things.

I am simply saying that many people trust DNS a lot more than they should in their protocols,
more so when they feel that they have DNSSEC turned on internally and they are going to an
internal address that they can "trust".  By exposing YARN through DNS it did not make it any
less secure, it just made it a lot simpler for someone to deploy something that is insecure.

> [Umbrella] Simplified discovery of services via DNS mechanisms
> --------------------------------------------------------------
>                 Key: YARN-4757
>                 URL: https://issues.apache.org/jira/browse/YARN-4757
>             Project: Hadoop YARN
>          Issue Type: New Feature
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Jonathan Maron
>         Attachments: YARN-4757- Simplified discovery of services via DNS mechanisms.pdf
> [See overview doc at YARN-4692, copying the sub-section ( to track all related
> In addition to completing the present story of service­-registry (YARN-913), we also
need to simplify the access to the registry entries. The existing read mechanisms of the YARN
Service Registry are currently limited to a registry specific (java) API and a REST interface.
In practice, this makes it very difficult for wiring up existing clients and services. For
e.g, dynamic configuration of dependent end­points of a service is not easy to implement
using the present registry­-read mechanisms, *without* code-changes to existing services.
> A good solution to this is to expose the registry information through a more generic
and widely used discovery mechanism: DNS. Service Discovery via DNS uses the well-­known
DNS interfaces to browse the network for services. YARN-913 in fact talked about such a DNS
based mechanism but left it as a future task. (Task) Having the registry information exposed
via DNS simplifies the life of services.

This message was sent by Atlassian JIRA

View raw message