cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrija Panic <andrija.pa...@gmail.com>
Subject Re: KVM HA fails under multiple management services
Date Mon, 24 Jun 2019 09:01:22 GMT
Li,

please test with  Indirect.agent.lb.check.interval=60 or similar, not 0
(zero), since that means it won't reconnect - this should solve your
concern.

As for the what is in what rack, it is your responsibility to disperse
infrastructure components appropriately, i.e. across racks and such.
We can't handle every case in that regards, hope you understand

Andrija

On Mon, 24 Jun 2019 at 02:24, li jerry <div8cn@hotmail.com> wrote:

> Thank you Nicolas and Andrija.
>
> Even if indirect.agent.lb.algorithm is configured as roundrobin, the
> probability of failure can only be reduced. But it does not solve 100% of
> the failure of KVM HA;
>
> Because in extreme cases, the management server and the kvm host may fail
> at the same time (for example, the management server and the KVM HOST are
> placed in the same rack, and the RACK will fail at the same time after the
> power failure)
>
>
> E.g;
>
> H1 is assigned and connected to M2
> H2 is assigned and connected to M3
> H3 is assigned and connected to M1
>
> When H1 and M2 fail simultaneously, HOST HA of H1 will be invalid;
>
> Should we have other protection mechanisms to avoid this?
>
> 发件人: Nicolas Vazquez<mailto:Nicolas.Vazquez@shapeblue.com>
> 发送时间: 2019年6月23日 23:31
> 收件人: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>;
> users<mailto:users@cloudstack.apache.org>
> 抄送: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>
> 主题: Re: KVM HA fails under multiple management services
>
> As Andrija mentioned that is expected behavior as the global setting is
> 'static'. It is also expected that your agents connect to the next
> management server on the 'host' list once the management server they are
> connected to is down.
> You can find more information of this feature on this link:
> https://www.shapeblue.com/software-based-agent-lb-for-cloudstack/
>
> Please note this is a different feature than host HA, in which CloudStack
> will try to recover hosts which are off via ipmi
>
> Obtener Outlook para Android<https://aka.ms/ghei36>
>
>
>
> De: Andrija Panic
> Enviado: domingo, 23 de junio 11:03
> Asunto: Re: KVM HA fails under multiple management services
> Para: users
> Cc: dev@cloudstack.apache.org
>
>
> Li,
>
> based on the Global Setting description for those 2, I would say that is
> the expected behaviour.
> i.e. change Indirect.agent.lb.check.interval to some other value, since 0
> means "don't check, don't reconnect" per what I read.
>
> Also, you might want to change from  Indirect.agent.lb.algorithm=static to
> some other value, since static means all your KVM agents will always
> connect to that one mgmt host that is the first one in the in the "host"
> list.
>
> Regards,
> Andrija
>
>
> Nicolas.Vazquez@shapeblue.com
> www.shapeblue.com<http://www.shapeblue.com>
> Amadeus House, Floral Street, London  WC2E 9DPUK
> @shapeblue
>
>
>
> On Sat, 22 Jun 2019 at 06:19, li jerry <div8cn@hotmail.com> wrote:
>
> >
> > Hello everyone
> > I recently tested the multiple management services, based on agent lb
> HOST
> > HA (KVM). It was found that in extreme cases, HA would fail; the details
> > are as follows:
> >
> >
> > Two management nodes, M1 (172.17.1.141) and M2 (172.17.1.142), share an
> > external database cluster
> > Three KVM nodes, H1, H2, H3
> > An external NFS primary storage
> >
> >
> > CLOUDSTACK parameter configuration
> > Indirect.agent.lb.algorithm=static
> > Indirect.agent.lb.check.interval=0
> > host=172.17.1.141,172.17.1.142
> >
> >
> > Through the agent.log analysis, all kvm agents are connected to the first
> > selection management node M1 (172.17.1.141):
> >
> > INFO [cloud.agent.Agent] (agentRequest-Handler-1:null) (logid:b30323e4)
> > Processed new management server list: 172.17.1.141,172.17.1.142@static
> >
> >
> >
> > In extreme cases:
> > KVM HOST and the preferred management server fail at the same time, KVM
> > HOST will not trigger HA detection
> >
> > E.g:
> >
> > M1+H1, power off at the same time; the state of H1 remains Disconnected,
> > and all VMs on H1 will not restart on other KVM nodes;
> > M1+H2, power off at the same time; the state of H1 remains Disconnected,
> > and all VMs on H2 will not restart on other KVM nodes;
> > M1+H3, power off at the same time; the state of H1 remains Disconnected,
> > and all VMs on H3 will not restart on other KVM nodes;
> >
>
>
> --
>
> Andrija Panić
>
>
>

-- 

Andrija Panić

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message