Mailing-List: contact dev-help@hawq.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hawq.incubator.apache.org
MIME-Version: 1.0
In-Reply-To: <CAH=zMbowGYpA=+maGqsFhB1aZsTe1bHgCub9SaMpH2j7vCpRGw@mail.gmail.com>
References: <CAHVwNMi25XCF7YzXqeg6G1oqRfAHy5csyAxx4ZUJkLzrUfJtzA@mail.gmail.com>
 <CAE44UQcWX33ik89hv1jWRKB_kpZrv3Qtpy9n6c-fs417uZTD4Q@mail.gmail.com>
 <CAHVwNMi8Rg5NG9ybD6cWypd00WxJaFhUECd5y3QzCmgBaVzQDw@mail.gmail.com>
 <CAE44UQfZxpwTfXKTsHuV1xEfCQg3dPtK+UmXkHXwoh_Mx53BGw@mail.gmail.com>
 <CAHVwNMirQRigr2bbQ2yRqbOGO4__0+R4ircMJ9yzPQMOQdpyjA@mail.gmail.com>
 <CAHVwNMijs2A8-g0nT2-5mdPyyycapRdDoqKPbKM_sr7djy6X0Q@mail.gmail.com> <CAH=zMbowGYpA=+maGqsFhB1aZsTe1bHgCub9SaMpH2j7vCpRGw@mail.gmail.com>
From: Shubham Sharma <ssharma@pivotal.io>
Date: Mon, 13 Nov 2017 19:06:08 -0800
Message-ID: <CAHVwNMhsLyUt+5yCwiajMnwf0GFmzGAk=SSWLRpXnUfnBUiHsw@mail.gmail.com>
Subject: Re: Hawq standby sync fails if there are existing connections to master
To: dev@hawq.incubator.apache.org
Content-Type: multipart/alternative; boundary="f40304361532247a1e055de8abb3"
archived-at: Tue, 14 Nov 2017 03:06:22 -0000

--f40304361532247a1e055de8abb3
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Thanks Radar, fixed this under PR -
https://github.com/apache/incubator-hawq/pull/1311

On Mon, Nov 13, 2017 at 6:32 PM, Radar Lei <rlei@pivotal.io> wrote:

> Hi Shubham,
>
> The behavior is intentional but use '-M' option to force it should be goo=
d.
> Thanks.
>
> Regards,
> Radar
>
> On Tue, Nov 14, 2017 at 10:05 AM, Shubham Sharma <ssharma@pivotal.io>
> wrote:
>
> > To close this thread, the source of this issue appears to be
> > tools/bin/hawq_ctl method _resync_standby. When this is called the
> command
> > formation does not include stop_mode options as passed to the arguments=
.
> >
> >  def _resync_standby(self):
> >         logger.info("Re-sync standby")
> >         cmd =3D "%s; hawq stop master -a;" % source_hawq_env
> >         check_return_code(local_ssh(cmd, logger), logger, "Stop hawq
> > cluster failed, exit")
> >         ......
> >         ......
> >
> > JIRA ID - HAWQ-1549
> >
> > On Mon, Nov 13, 2017 at 5:56 PM, Shubham Sharma <ssharma@pivotal.io>
> > wrote:
> >
> > > Thanks, I think I found out where the issue is. Will create a JIRA an=
d
> > > submit PR.
> > >
> > > On Mon, Nov 13, 2017 at 5:54 PM, Lei Chang <chang.lei.cn@gmail.com>
> > wrote:
> > >
> > >> oh, yes, make sense, I overlooked the -M option here.
> > >>
> > >> Here we should respect the input "-M" option.
> > >>
> > >> Cheers
> > >> Lei
> > >>
> > >>
> > >>
> > >> On Tue, Nov 14, 2017 at 8:46 AM, Shubham Sharma <ssharma@pivotal.io>
> > >> wrote:
> > >>
> > >> > Thanks for the response Lei.
> > >> >
> > >> > I understand this completely and agree with the behavior that we
> > should
> > >> not
> > >> > brutally terminate connections.
> > >> >
> > >> > However, the case here is, am deliberately trying to use stop mode
> > using
> > >> > command `hawq init standby -n -v -M fast`. At this point as a user=
 I
> > >> > understand that connections will be terminated without warning. Wh=
en
> > >> > passing option `-M fast` hawq should not interrupt me.
> > >> >
> > >> > Basically, hawq command line is not respecting `-M fast`. Whereas
> hawq
> > >> init
> > >> > standby --help documents this option. Let me know if this makes
> sense.
> > >> >
> > >> >
> > >> > hawq init standby --help
> > >> >
> > >> > Usage: HAWQ management scripts options
> > >> >
> > >> >
> > >> > Options:
> > >> >
> > >> >   -h, --help            show this help message and exit
> > >> >
> > >> >   -a, --prompt          Execute automatically
> > >> >
> > >> >   -M STOP_MODE, --mode=3DSTOP_MODE
> > >> >
> > >> >                         HAWQ stop mode: smart/fast/immediate
> > >> >
> > >> >
> > >> > On Mon, Nov 13, 2017 at 4:32 PM, Lei Chang <chang.lei.cn@gmail.com=
>
> > >> wrote:
> > >> >
> > >> > > Hi Shubham,
> > >> > >
> > >> > > The behavior is intentional. If there are connections when HAWQ
> init
> > >> > > standby, it is better to warn the client instead of cutting the
> > >> > connections
> > >> > > brutally.
> > >> > >
> > >> > > Cheers
> > >> > > Lei
> > >> > >
> > >> > >
> > >> > >
> > >> > > On Mon, Nov 13, 2017 at 11:58 AM, Shubham Sharma <
> > ssharma@pivotal.io>
> > >> > > wrote:
> > >> > >
> > >> > > > Hello folks,
> > >> > > >
> > >> > > > Recently observed a behaviour while re-syncing standby from ha=
wq
> > >> > command
> > >> > > > line.
> > >> > > >
> > >> > > > Here are the reproduction steps -
> > >> > > >
> > >> > > > 1 - Open a client connection to hawq using psql
> > >> > > > 2 - From a different terminal run command - hawq init standby =
-n
> > -v
> > >> -M
> > >> > > fast
> > >> > > > 3 - Standby resync fails with error
> > >> > > >
> > >> > > >
> > >> > > > 20171113:03:49:21:158354 hawq_stop:hdp3:gpadmin-[
> WARNING]:-There
> > >> are
> > >> > > > other connections to this instance, shutdown mode smart aborte=
d
> > >> > > >
> > >> > > > 20171113:03:49:21:158354 hawq_stop:hdp3:gpadmin-[
> WARNING]:-Either
> > >> > > > remove connections, or use 'hawq stop master -M fast' or 'hawq
> > stop
> > >> > > > master -M immediate'
> > >> > > >
> > >> > > > 20171113:03:49:21:158354 hawq_stop:hdp3:gpadmin-[WARNING]:-See
> > hawq
> > >> > > > stop --help for all options
> > >> > > >
> > >> > > > 20171113:03:49:21:158354 hawq_stop:hdp3:gpadmin-[ERROR]:-Activ=
e
> > >> > > > connections. Aborting shutdown...
> > >> > > >
> > >> > > > 20171113:03:49:21:158143 hawq_init:hdp3:gpadmin-[ERROR]:-Stop
> > hawq
> > >> > > > cluster failed, exit
> > >> > > >
> > >> > > > 4 - My understanding is when -M (stop mode) is passed it shoul=
d
> > >> > terminate
> > >> > > > existing client connections. Also, it seems like a good practi=
ce
> > to
> > >> > > > terminate client connections before standby master resync.
> > >> > > >
> > >> > > > Is this an expected behavior in hawq ? If not, I can open a JI=
RA
> > and
> > >> > work
> > >> > > > on a pull request to fix this.
> > >> > > >
> > >> > > > Looking forward to your thoughts on this.
> > >> > > > =E2=80=8B
> > >> > > > --
> > >> > > > Regards,
> > >> > > > Shubham Sharma
> > >> > > >
> > >> > >
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > Regards,
> > >> > Shubham Sharma
> > >> > Staff Customer Engineer
> > >> > Pivotal Global Support Services
> > >> > ssharma@pivotal.io
> > >> > Direct Tel: +1(510)-304-8201
> > >> > Office Hours: Mon-Fri 9:00 am to 5:00 pm PDT
> > >> > Out of Office Hours Contact +1 877-477-2269
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > Regards,
> > > Shubham Sharma
> > > Staff Customer Engineer
> > > Pivotal Global Support Services
> > > ssharma@pivotal.io
> > > Direct Tel: +1(510)-304-8201 <(510)%20304-8201>
> > > Office Hours: Mon-Fri 9:00 am to 5:00 pm PDT
> > > Out of Office Hours Contact +1 877-477-2269 <(877)%20477-2269>
> > >
> >
> >
> >
> > --
> > Regards,
> > Shubham Sharma
> > Staff Customer Engineer
> > Pivotal Global Support Services
> > ssharma@pivotal.io
> > Direct Tel: +1(510)-304-8201
> > Office Hours: Mon-Fri 9:00 am to 5:00 pm PDT
> > Out of Office Hours Contact +1 877-477-2269
> >
>


--=20
Regards,
Shubham Sharma
Staff Customer Engineer
Pivotal Global Support Services
ssharma@pivotal.io
Direct Tel: +1(510)-304-8201
Office Hours: Mon-Fri 9:00 am to 5:00 pm PDT
Out of Office Hours Contact +1 877-477-2269

--f40304361532247a1e055de8abb3--