hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Joseph Evans (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4757) [Umbrella] Simplified discovery of services via DNS mechanisms
Date Mon, 14 Mar 2016 16:30:33 GMT

    [ https://issues.apache.org/jira/browse/YARN-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15193577#comment-15193577

Robert Joseph Evans commented on YARN-4757:

I am +1 on the idea of using DNS for long lived service discovery, but we need to be very
very careful about security.  If we are not all of the problems possible with https://en.wikipedia.org/wiki/DNS_spoofing
would likely be possible with this too.  We need to be positive that we can restrict the names
allowed so there are no conflicts with other servers on the network/internet.  Additionally
if we make this super simple, which is the entire goal here, then we are covering up some
really potentially serious issues with client code, that a normal server running off YARN
would not expect to have.  It really comes down to any service running on YARN that wants
to be secure needs to have 2 way authentication client authenticates server and server authenticates
clients.  There are timing attacks and other things that can happen when a process crashes
and lets go of a port.  Internal web services especially feel vulnerable because unless you
enable SSL they will be insecure, something that many groups avoid on internal services because
of the extra overhead of doing encryption.

Do you plan on handling ephemeral ports in some way? As far as I know there is no standard
for including port(s) in a DNS entry.  If we do come up with something that is non-standard
doesn't that still necessitate client side changes which was an expressed goal of this JIRA?
 If we don't handle ephemeral ports are we going to add in mesos-like scheduling of ports?


> [Umbrella] Simplified discovery of services via DNS mechanisms
> --------------------------------------------------------------
>                 Key: YARN-4757
>                 URL: https://issues.apache.org/jira/browse/YARN-4757
>             Project: Hadoop YARN
>          Issue Type: New Feature
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Jonathan Maron
> [See overview doc at YARN-4692, copying the sub-section ( to track all related
> In addition to completing the present story of service­-registry (YARN-913), we also
need to simplify the access to the registry entries. The existing read mechanisms of the YARN
Service Registry are currently limited to a registry specific (java) API and a REST interface.
In practice, this makes it very difficult for wiring up existing clients and services. For
e.g, dynamic configuration of dependent end­points of a service is not easy to implement
using the present registry­-read mechanisms, *without* code-changes to existing services.
> A good solution to this is to expose the registry information through a more generic
and widely used discovery mechanism: DNS. Service Discovery via DNS uses the well-­known
DNS interfaces to browse the network for services. YARN-913 in fact talked about such a DNS
based mechanism but left it as a future task. (Task) Having the registry information exposed
via DNS simplifies the life of services.

This message was sent by Atlassian JIRA

View raw message