cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Remi Bergsma <>
Subject Re: [DISCUSS] Out of Band VR migration, should we reboot VR or not?
Date Wed, 03 Jun 2015 18:34:37 GMT
Hi all,

I just had a look at this more closely and had a chat with Ian about it. The only way for
the original problem to happen (losing iptables rules) is if the live migrate would fail and
the hypervisor rebooted the vm. The cause is the non-persistance of the router configuration,
which is fixed in 4.6 by the way. I would say failing live migrations does not happen often
(I have never seen it happening).

Anyway, once this happens to the router, it is stuck in a state where it does not have the
linklocal configuration any more. Would CloudStack be able to issue a aggregate command while
it cannot reach it? Rebooting might be the only way out after all. It’s just that rebooting
by default in case of out-of-band migrations I’d say is a little bit too much. CloudStack
would detect the problem anyway, as it cannot reach the linklocal anymore.

The interesting situation is that we have releases out there that now reboot by default.

My proposal to solve it in 4.4 and 4.5:
- Implement a setting ‘reboot systemvm when out-of-band migration detected’.
The default should be false and release notes should mention a changed behaviour from 4.4.3
and 4.5.1. To get the old behaviour, switch to true. A small group of people should be interested
in this.

I guess this is the best of both worlds. Do you guys agree?

The other option I see is to revert the commit, as I think that serves most people.

Who is willing to help implement it?

Regards, Remi

On 03 Jun 2015, at 17:42, Rene Moser <<>>


On 03.06.2015 17:06, Ian Southam wrote:
If the machine crashes and/or rebooted during the oob migration by a party that is not the
orchestrator, (read vCenter) then the rules will be lost.

I fully agree, a reboot due a failing live migration, would cause a
reboot. So what? Then we blame VMware, the orchestration will reboot the
VR and everything is fine. This would cause seconds of outage.

But then the missing persistence of the iptables would be the problem,
not the live migration task, right?

We should fix the persistence of the rules during reboot and not try to
be more clever then the hypervisor cluster orchestration.

Just my 2 cents.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message