hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Allen Wittenauer ...@effectivemachines.com>
Subject Re: [VOTE] Merge yarn-native-services branch into trunk
Date Tue, 05 Sep 2017 22:11:50 GMT

> On Sep 5, 2017, at 2:53 PM, Jian He <jhe@hortonworks.com> wrote:
> 
>> Based on the documentation, this doesn’t appear to be a fully function DNS server
as an admin would expect (e.g., BIND, Knot, whatever).  Where’s forwarding? How do I setup
notify? Are secondaries even supported? etc, etc.
> 
> It seems like this is a rehash of some of the discussion you and others had on the JIRA.
The DNS here is a thin layer backed by service registry. My understanding from the JIRA is
that there are no claims that this is already a DNS with all the bells and whistles - its
goal is mainly to expose dynamic services running on YARN as end-points. Clearly, this is
an optional daemon, if the provided feature set is deemed insufficient, an alternative solution
can be plugged in by specific admins because the DNS piece is completely decoupled from the
rest of native-services. 

	If it doesn’t have all the bells and whistles, then it shouldn’t be on port 53 by default.
It should also be documented that one *can’t* do these things.  If the standard config is
likely to be a “real” server on port 53 either acting as a secondary to the YARN one or
at least able to forward queries to it, then these need to get documented.  As it stands,
operations folks are going to be taken completely by surprise by some relatively random process
sitting on a very well established port.

>> In fact:  was this even tested on port 53? How does this get launched such that it
even has access to open port 53?  I don’t see any calls to use the secure daemon code in
the shell scripts. Is there any jsvc voodoo or is it just “run X as root”?
> 
> Yes, we have tested this DNS server on port 53 on a cluster by running the DNS server
as root user. The port is clearly configurable, so the admin has two options. Run as root
+ port 53. Run as non-root + non-privileged port. We tested and left it as port 53 to keep
it on a standard DNS port. It is already documented as such though I can see that part can
be improved a little.

	*how* is it getting launched on a privileged port? It sounds like the expectation is to run
“command” as root.   *ALL* of the previous daemons in Hadoop that needed a privileged
port used jsvc.  Why isn’t this one? These questions matter from a security standpoint.
 

>> 	4) Post-merge, yarn usage information is broken.  This is especially bad since it
doesn’t appear that YarnCommands was ever updated to include the new sub-commands.
> 
> The “yarn” usage command is working for me. what do you mean ? 

Check the output.  It’s pretty obviously borked:

===snip====

    Daemon Commands:

nodemanager          run a nodemanager on each worker
proxyserver          run the web app proxy server
resourcemanager      run the ResourceManager
router               run the Router daemon
timelineserver       run the timeline server

    Run a service Commands:

service              run a service

    Run yarn-native-service rest server Commands:

apiserver            run yarn-native-service rest server


===snip===

> Yeah, looks like some previous features also forgot to update YarnCommands.md for the
new sub commands 

	Likely.  But I was actually interested in playing with this one to compare it to the competition.
 [Lucky you. ;) ]  But with pretty much zero documentation….



---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org


Mime
View raw message