Mailing-List: contact yarn-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
Subject: Re: [VOTE] Merge yarn-native-services branch into trunk
From: Allen Wittenauer <aw@effectivemachines.com>
In-Reply-To: <550243F0-A333-4D27-90CE-D926162BA442@hortonworks.com>
Date: Tue, 5 Sep 2017 15:11:50 -0700
Cc: "yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>,
 Hadoop Common <common-dev@hadoop.apache.org>,
 Hdfs-dev <hdfs-dev@hadoop.apache.org>,
 "mapreduce-dev@hadoop.apache.org" <mapreduce-dev@hadoop.apache.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <980A1DD9-467B-4DB0-88DD-7EC1E1E3E7B8@effectivemachines.com>
References: <B948E143-8202-4043-BA8F-0360B6AB4425@hortonworks.com> <1FEA8CAF-0E6C-4050-9DAC-68963681027B@effectivemachines.com> <550243F0-A333-4D27-90CE-D926162BA442@hortonworks.com>
To: Jian He <jhe@hortonworks.com>
archived-at: Tue, 05 Sep 2017 22:12:07 -0000


> On Sep 5, 2017, at 2:53 PM, Jian He <jhe@hortonworks.com> wrote:
>=20
>> Based on the documentation, this doesn=E2=80=99t appear to be a fully =
function DNS server as an admin would expect (e.g., BIND, Knot, =
whatever).  Where=E2=80=99s forwarding? How do I setup notify? Are =
secondaries even supported? etc, etc.
>=20
> It seems like this is a rehash of some of the discussion you and =
others had on the JIRA. The DNS here is a thin layer backed by service =
registry. My understanding from the JIRA is that there are no claims =
that this is already a DNS with all the bells and whistles - its goal is =
mainly to expose dynamic services running on YARN as end-points. =
Clearly, this is an optional daemon, if the provided feature set is =
deemed insufficient, an alternative solution can be plugged in by =
specific admins because the DNS piece is completely decoupled from the =
rest of native-services.=20

	If it doesn=E2=80=99t have all the bells and whistles, then it =
shouldn=E2=80=99t be on port 53 by default. It should also be documented =
that one *can=E2=80=99t* do these things.  If the standard config is =
likely to be a =E2=80=9Creal=E2=80=9D server on port 53 either acting as =
a secondary to the YARN one or at least able to forward queries to it, =
then these need to get documented.  As it stands, operations folks are =
going to be taken completely by surprise by some relatively random =
process sitting on a very well established port.

>> In fact:  was this even tested on port 53? How does this get launched =
such that it even has access to open port 53?  I don=E2=80=99t see any =
calls to use the secure daemon code in the shell scripts. Is there any =
jsvc voodoo or is it just =E2=80=9Crun X as root=E2=80=9D?
>=20
> Yes, we have tested this DNS server on port 53 on a cluster by running =
the DNS server as root user. The port is clearly configurable, so the =
admin has two options. Run as root + port 53. Run as non-root + =
non-privileged port. We tested and left it as port 53 to keep it on a =
standard DNS port. It is already documented as such though I can see =
that part can be improved a little.

	*how* is it getting launched on a privileged port? It sounds =
like the expectation is to run =E2=80=9Ccommand=E2=80=9D as root.   =
*ALL* of the previous daemons in Hadoop that needed a privileged port =
used jsvc.  Why isn=E2=80=99t this one? These questions matter from a =
security standpoint. =20

>> 	4) Post-merge, yarn usage information is broken.  This is =
especially bad since it doesn=E2=80=99t appear that YarnCommands was =
ever updated to include the new sub-commands.
>=20
> The =E2=80=9Cyarn=E2=80=9D usage command is working for me. what do =
you mean ?=20

Check the output.  It=E2=80=99s pretty obviously borked:

=3D=3D=3Dsnip=3D=3D=3D=3D

    Daemon Commands:

nodemanager          run a nodemanager on each worker
proxyserver          run the web app proxy server
resourcemanager      run the ResourceManager
router               run the Router daemon
timelineserver       run the timeline server

    Run a service Commands:

service              run a service

    Run yarn-native-service rest server Commands:

apiserver            run yarn-native-service rest server


=3D=3D=3Dsnip=3D=3D=3D

> Yeah, looks like some previous features also forgot to update =
YarnCommands.md for the new sub commands=20

	Likely.  But I was actually interested in playing with this one =
to compare it to the competition.  [Lucky you. ;) ]  But with pretty =
much zero documentation=E2=80=A6.


---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org