hawq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Radar Lei <r...@pivotal.io>
Subject Re: Hawq standby sync fails if there are existing connections to master
Date Tue, 14 Nov 2017 02:32:33 GMT
Hi Shubham,

The behavior is intentional but use '-M' option to force it should be good.
Thanks.

Regards,
Radar

On Tue, Nov 14, 2017 at 10:05 AM, Shubham Sharma <ssharma@pivotal.io> wrote:

> To close this thread, the source of this issue appears to be
> tools/bin/hawq_ctl method _resync_standby. When this is called the command
> formation does not include stop_mode options as passed to the arguments.
>
>  def _resync_standby(self):
>         logger.info("Re-sync standby")
>         cmd = "%s; hawq stop master -a;" % source_hawq_env
>         check_return_code(local_ssh(cmd, logger), logger, "Stop hawq
> cluster failed, exit")
>         ......
>         ......
>
> JIRA ID - HAWQ-1549
>
> On Mon, Nov 13, 2017 at 5:56 PM, Shubham Sharma <ssharma@pivotal.io>
> wrote:
>
> > Thanks, I think I found out where the issue is. Will create a JIRA and
> > submit PR.
> >
> > On Mon, Nov 13, 2017 at 5:54 PM, Lei Chang <chang.lei.cn@gmail.com>
> wrote:
> >
> >> oh, yes, make sense, I overlooked the -M option here.
> >>
> >> Here we should respect the input "-M" option.
> >>
> >> Cheers
> >> Lei
> >>
> >>
> >>
> >> On Tue, Nov 14, 2017 at 8:46 AM, Shubham Sharma <ssharma@pivotal.io>
> >> wrote:
> >>
> >> > Thanks for the response Lei.
> >> >
> >> > I understand this completely and agree with the behavior that we
> should
> >> not
> >> > brutally terminate connections.
> >> >
> >> > However, the case here is, am deliberately trying to use stop mode
> using
> >> > command `hawq init standby -n -v -M fast`. At this point as a user I
> >> > understand that connections will be terminated without warning. When
> >> > passing option `-M fast` hawq should not interrupt me.
> >> >
> >> > Basically, hawq command line is not respecting `-M fast`. Whereas hawq
> >> init
> >> > standby --help documents this option. Let me know if this makes sense.
> >> >
> >> >
> >> > hawq init standby --help
> >> >
> >> > Usage: HAWQ management scripts options
> >> >
> >> >
> >> > Options:
> >> >
> >> >   -h, --help            show this help message and exit
> >> >
> >> >   -a, --prompt          Execute automatically
> >> >
> >> >   -M STOP_MODE, --mode=STOP_MODE
> >> >
> >> >                         HAWQ stop mode: smart/fast/immediate
> >> >
> >> >
> >> > On Mon, Nov 13, 2017 at 4:32 PM, Lei Chang <chang.lei.cn@gmail.com>
> >> wrote:
> >> >
> >> > > Hi Shubham,
> >> > >
> >> > > The behavior is intentional. If there are connections when HAWQ init
> >> > > standby, it is better to warn the client instead of cutting the
> >> > connections
> >> > > brutally.
> >> > >
> >> > > Cheers
> >> > > Lei
> >> > >
> >> > >
> >> > >
> >> > > On Mon, Nov 13, 2017 at 11:58 AM, Shubham Sharma <
> ssharma@pivotal.io>
> >> > > wrote:
> >> > >
> >> > > > Hello folks,
> >> > > >
> >> > > > Recently observed a behaviour while re-syncing standby from hawq
> >> > command
> >> > > > line.
> >> > > >
> >> > > > Here are the reproduction steps -
> >> > > >
> >> > > > 1 - Open a client connection to hawq using psql
> >> > > > 2 - From a different terminal run command - hawq init standby
-n
> -v
> >> -M
> >> > > fast
> >> > > > 3 - Standby resync fails with error
> >> > > >
> >> > > >
> >> > > > 20171113:03:49:21:158354 hawq_stop:hdp3:gpadmin-[WARNING]:-There
> >> are
> >> > > > other connections to this instance, shutdown mode smart aborted
> >> > > >
> >> > > > 20171113:03:49:21:158354 hawq_stop:hdp3:gpadmin-[WARNING]:-Either
> >> > > > remove connections, or use 'hawq stop master -M fast' or 'hawq
> stop
> >> > > > master -M immediate'
> >> > > >
> >> > > > 20171113:03:49:21:158354 hawq_stop:hdp3:gpadmin-[WARNING]:-See
> hawq
> >> > > > stop --help for all options
> >> > > >
> >> > > > 20171113:03:49:21:158354 hawq_stop:hdp3:gpadmin-[ERROR]:-Active
> >> > > > connections. Aborting shutdown...
> >> > > >
> >> > > > 20171113:03:49:21:158143 hawq_init:hdp3:gpadmin-[ERROR]:-Stop
> hawq
> >> > > > cluster failed, exit
> >> > > >
> >> > > > 4 - My understanding is when -M (stop mode) is passed it should
> >> > terminate
> >> > > > existing client connections. Also, it seems like a good practice
> to
> >> > > > terminate client connections before standby master resync.
> >> > > >
> >> > > > Is this an expected behavior in hawq ? If not, I can open a JIRA
> and
> >> > work
> >> > > > on a pull request to fix this.
> >> > > >
> >> > > > Looking forward to your thoughts on this.
> >> > > > ‚Äč
> >> > > > --
> >> > > > Regards,
> >> > > > Shubham Sharma
> >> > > >
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > Regards,
> >> > Shubham Sharma
> >> > Staff Customer Engineer
> >> > Pivotal Global Support Services
> >> > ssharma@pivotal.io
> >> > Direct Tel: +1(510)-304-8201
> >> > Office Hours: Mon-Fri 9:00 am to 5:00 pm PDT
> >> > Out of Office Hours Contact +1 877-477-2269
> >> >
> >>
> >
> >
> >
> > --
> > Regards,
> > Shubham Sharma
> > Staff Customer Engineer
> > Pivotal Global Support Services
> > ssharma@pivotal.io
> > Direct Tel: +1(510)-304-8201 <(510)%20304-8201>
> > Office Hours: Mon-Fri 9:00 am to 5:00 pm PDT
> > Out of Office Hours Contact +1 877-477-2269 <(877)%20477-2269>
> >
>
>
>
> --
> Regards,
> Shubham Sharma
> Staff Customer Engineer
> Pivotal Global Support Services
> ssharma@pivotal.io
> Direct Tel: +1(510)-304-8201
> Office Hours: Mon-Fri 9:00 am to 5:00 pm PDT
> Out of Office Hours Contact +1 877-477-2269
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message