cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simon Weller <swel...@ena.com>
Subject Re: database high availability question vs haproxy
Date Fri, 05 Jun 2015 14:28:52 GMT
I'm not sure whether that's configurable and  I haven't seen any examples out there that point
to it being configurable. Having said that, there are people a lot more familiar with that
than I am on the list.

One option you do have for this particular failure situation, is to use monit to monitor the
pid of the CS management service and if it's not running, force it to start up again.

Losing a load balancer completely is very rare, so it's not likely to happen often at all.

- Si
________________________________________
From: Andrija Panic <andrija.panic@gmail.com>
Sent: Friday, June 5, 2015 9:07 AM
To: users@cloudstack.apache.org
Cc: dev@cloudstack.apache.org
Subject: Re: database high availability question vs haproxy

Exactly...thanks Simon for the time and help :)

Is there any mysql timeout/retry parameter that is set in db.properties
(like db.cloud.queriesBeforeRetryMaster=5000), but I'm wondering since I'm
using keepalived/haproxy setup, it takes up to 5-6 sec for keepalive to
detect haproxy is down (while testing I shutdown haproxu on active node),
and then keepalvied moves IP to another node - meaning I have 5-6sec of no
connection between ACS mgmt servers and database/haproxy.

Is this timeout configurabile on ACS/mysql connector side ?

Thanks again,
Andrija

On 5 June 2015 at 15:54, Simon Weller <sweller@ena.com> wrote:

> I think haproxy is a better design in my opinion. You're going to have to
> use haproxy to balance the host agents to multiple management servers
> anyway, so you'll already be using it. haproxy can then manage the health
> checks to Galera rather than over complicating it with the CS Management
> configuration.
>
> - Si
>
> ________________________________________
> From: Andrija Panic <andrija.panic@gmail.com>
> Sent: Friday, June 5, 2015 8:37 AM
> To: users@cloudstack.apache.org
> Cc: dev@cloudstack.apache.org
> Subject: Re: database high availability question vs haproxy
>
> Thanks Simon - that is what I thought...
>
> So my question would be then, haproxy vs native ACS/mysql connector going
> to galera1/galera2/etc...will figure out, for now we use haproxy for
> mysql/galera loadbalancing...
>
>
> THanks a lot Simon,
> Andrija
>
> On 5 June 2015 at 15:18, Simon Weller <sweller@ena.com> wrote:
>
> >
> > Personally, I think that Gallera is always going to be a safer option, as
> > it handles conflict resolution natively. Having said that, it appears
> care
> > has been taken in designing the ACS MGMT DB integration so that the
> chance
> > of conflicts is very low. Galera requires a 3 nodes minimum, so it's a
> lot
> > of hardware unless you've got plans to use it elsewhere in your
> > organisation.
> >
> > The downside to Galera, is that it's synchronous replication, so it needs
> > very low latency between nodes. That doesn't make it a good candidate for
> > geographic separation between DB nodes for a DR scenario.
> >
> > You're understanding of the replication structure, as based on the design
> > document is correct. MySQL (or Galera) handles all the replication. ACS
> > just handles which node it's writing and reading from. In a 2 node native
> > MySQL cluster, it's expected that you are setup for cross master-master
> > replication.
> >
> > - Si
> > ________________________________________
> > From: Andrija Panic <andrija.panic@gmail.com>
> > Sent: Friday, June 5, 2015 2:41 AM
> > To: dev@cloudstack.apache.org
> > Cc: users@cloudstack.apache.org
> > Subject: Re: database high availability question vs haproxy
> >
> > Hi Simon,
> >
> > thanks for the link - actually I have already read this - but Im still
> > seaking for some answeres :) :
> >
> > - real world experience with DB HA in general  - is i better to use
> > haproxy(clustered/redudant) for mysql towards Galera cluster - or simply
> to
> > reference 2 nodes (1 as master, another as slave) with native ACS DB HA
> -
> > silly question but anyway...
> > - my understanding - ACS just pings and connects to master or slave (all
> > replication etc, is done from my side, not from ACS) ?
> >
> > Thanks again and any info is greatly appreciated.
> >
> > Andrija
> >
> > On 4 June 2015 at 16:23, Simon Weller <sweller@ena.com> wrote:
> >
> > > Andrija,
> > >
> > > Here is the original design document, and it should give you a better
> > idea
> > > of what is implemented today:
> > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=34838207
> > >
> > > We have plans to test this in our lab soon, but just haven't got around
> > to
> > > it yet.
> > >
> > > - Si
> > >
> > > ________________________________________
> > > From: Andrija Panic <andrija.panic@gmail.com>
> > > Sent: Thursday, June 4, 2015 9:08 AM
> > > To: dev@cloudstack.apache.org; users@cloudstack.apache.org
> > > Subject: Re: database high availability question vs haproxy
> > >
> > > Anyone :) ?
> > >
> > > On 31 May 2015 at 00:26, Andrija Panic <andrija.panic@gmail.com>
> wrote:
> > >
> > > > Hi,
> > > >
> > > > I would have a question on database HA feature in db.properties (
> > > >
> > >
> >
> http://cloudstack-administration.readthedocs.org/en/latest/reliability.html#configuring-database-high-availability
> > > > )
> > > >
> > > > If I understand correctly, it is up to the admin to provide
> appropriate
> > > > mysql HA (active-active, galera, etc) and ACS management server will
> > > JUST
> > > > try to connect to slaves if the master is down ?
> > > >
> > > > We are running Galera, with haproxy/keepalived, and by using stoping
> > > > haproxy, it takes i.e. 6sec for keepalived to detect haproxy is down,
> > and
> > > > failover IP to another host.
> > > >
> > > > During these 6 seconds, ACS managemnt server goes dead, because of
> this
> > > DB
> > > > unavailability.
> > > >
> > > > So my wondering, is better to use ACS db HA feature, instead of
> > > > loadbalancer for this specific purpose ?
> > > > (we are also using haproxy/keepalived for management server
> > loadbalancing
> > > > - 2 servers in backend...)
> > > >
> > > > Any experience shared is really appreciated !
> > > > --
> > > >
> > > > Andrija Panić
> > > >
> > >
> > >
> > >
> > > --
> > >
> > > Andrija Panić
> > >
> >
> >
> >
> > --
> >
> > Andrija Panić
> >
>
>
>
> --
>
> Andrija Panić
>



--

Andrija Panić

Mime
View raw message