Mailing-List: contact dev-help@hawq.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hawq.incubator.apache.org
Date: Sat, 27 Feb 2016 22:33:02 -0800
From: Konstantin Boudnik <cos@apache.org>
To: dev@hawq.incubator.apache.org
Subject: Re: ssh'ing around the cluster
Message-ID: <20160228063302.GH3297@tpx>
Mail-Followup-To: dev@hawq.incubator.apache.org
References: <20160226014020.GT24478@tpx>
 <CAE44UQehN9dmKA1Mpyr_6QWgvh4cgQLPpKnh6eaJj29fV=hsYQ@mail.gmail.com>
 <20160226050350.GY24478@tpx>
 <20160227224447.GC3297@tpx>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="K1n7F7fSdjvFAEnM"
Content-Disposition: inline
In-Reply-To: <20160227224447.GC3297@tpx>
User-Agent: Mutt/1.5.21 (2010-09-15)

--K1n7F7fSdjvFAEnM
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Hi Lei.

Looks like in the current incarnation, Hawq isn't capable of running local
operations at all. I finally found the source of this issue, which took me a
while considering that I've dealt with Python three time in my lifetime,
including this one.  Anyway, hawq_ctl is _only_ operates with remote_ssh wh=
en
it gets to the init of different parts of the cluster. The function is taken
=66rom hawqpylib/hawqlib.py and runs ssh explicitly.

I propose to implement the functionality where local execution is indeed
possible, as I've described earlier. Thoughts?

Regards,
  Cos

On Sat, Feb 27, 2016 at 02:44PM, Konstantin Boudnik wrote:
> On Thu, Feb 25, 2016 at 09:03PM, Konstantin Boudnik wrote:
> > On Fri, Feb 26, 2016 at 10:53AM, Lei Chang wrote:
> > > Hi Konstantin,
> > >=20
> > > If I understand correctly, what you requested is in current hawq code=
 base.
> > > To init hawq, "hawq init cluster" is one way, you can also use "hawq =
init
> > > master" and "hawq init segment" on all cluster nodes. Master and segm=
ents
> > > are decoupled in 2.0.
> >=20
> > Ah, nice. Let me try that. Thanks!
>=20
> Well, unfortunately, it doesn't work as expected. Even as I do
>     $ sudo -u hawq /usr/bin/hawq init master
>=20
> I see that the process is always going to local_ssh call, which ends up i=
n an attempt to ssh into localhost.
> Traceback (most recent call last):
>   File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
>     "__main__", fname, loader, pkg_name)
>   File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
>     exec code in run_globals
>   File "/usr/lib64/python2.7/trace.py", line 819, in <module>
>     main()
>   File "/usr/lib64/python2.7/trace.py", line 807, in main
>     t.runctx(code, globs, globs)
>   File "/usr/lib64/python2.7/trace.py", line 513, in runctx
>     exec cmd in globals, locals
>   File "/usr/lib/hawq/bin/hawq", line 133, in <module>
>     main()
>   File "/usr/lib/hawq/bin/hawq", line 80, in main
>     result =3D local_ssh(cmd)
>   File "/usr/lib/hawq/bin/hawq", line 35, in local_ssh
>     result =3D subprocess.Popen(cmd, shell=3DTrue).wait()
>   File "/usr/lib64/python2.7/subprocess.py", line 1376, in wait
>     pid, sts =3D _eintr_retry_call(os.waitpid, self.pid, 0)
>   File "/usr/lib64/python2.7/subprocess.py", line 478, in _eintr_retry_ca=
ll
>     return func(*args)
>=20
> Any ideas what I am doing wrong? Thanks
>   Cos
>=20
> > > On Fri, Feb 26, 2016 at 9:40 AM, Konstantin Boudnik <cos@apache.org> =
wrote:
> > >=20
> > > > Guys,
> > > >
> > > > more revelations about the way Hawq is designed to work with the se=
rvice
> > > > bootstraps, config management, and so on. Looking into hawq_ctl and
> > > > observing
> > > > the behavior of 'hawq init cluster' I see that a number of operatio=
ns is
> > > > intended to be initiated from presumably, a master node, and carrie=
d on all
> > > > the nodes of a hawq cluster via the means or ssh (or rsync). While =
doing so
> > > > might be a convenient shortcut for the development environment, it =
isn't as
> > > > much in a real deployment. For one, it requires password-less SSH a=
ccess
> > > > between nodes, which isn't (generally speaking) how data centers mi=
ght
> > > > operate.
> > > >
> > > > Perhaps a better way of separating the concerns here is to have iso=
lated
> > > > functions to perform only local-node operations, and a wrapper to r=
un the
> > > > same
> > > > functionality on the all remote nodes, via ssh or else. If such spl=
it is
> > > > done,
> > > > then an orchestration mechanism (such as a state machine similar to=
 Puppet
> > > > or
> > > > Chef), would execute the scripts on separate nodes in full isolatio=
n. And
> > > > if
> > > > so desired in a dev environment, the current functionality would be
> > > > available
> > > > as well.
> > > >
> > > > Thoughts? Regards,
> > > >   Cos
> > > >
>=20
>=20


--K1n7F7fSdjvFAEnM
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iF4EAREIAAYFAlbSlJ4ACgkQenyFlstYjhJ7VgD/S1w+RFlVfB8qN+GrHxTnPR46
1ARb4VqcwbdL2gG+EDcBAJhA5gezFSH8b+MNBTcGKqH0zYnC58lj8eUVEytct4bv
=yA1N
-----END PGP SIGNATURE-----

--K1n7F7fSdjvFAEnM--