cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Garith Dugmore <>
Subject Re: HA issue and Xen resets
Date Thu, 23 Oct 2014 06:59:28 GMT
Hi Paul,

I chose a basic zone with all networks on one cable ie. I only have one cable plugged into
the xen server so yanking the cable should have stopped all comms to the machine. Thanks for
the note on xen rebuilds - will look into that.


----- Original Message -----
> From: "Paul Angus" <>
> To:
> Sent: Wednesday, October 22, 2014 5:41:51 PM
> Subject: RE: HA issue and Xen resets
> Hi Garith,
> There are a few questions wrapped up in there.
> WRT HA, there a number of reasons why CloudStack might not do anything as a
> result of a cable pull. For instance was the network cable you unplugged
> also carrying the primary storage? If not, then the XenServer will continue
> to run the VM and continue to send heartbeats to its shared storage.
> CloudStack would see that and decide that restarting the instance somewhere
> else would be dangerous as it could result in a split-brain.  You'll likely
> see in the CloudStack logs whether CloudStack has detected a problem and
> what it intends to do about it (if anything)
> It is possible to re-add a XenServer, but it's generally more trouble than
> it's worth, there's quite a bit of cleaning up to do and you never know what
> legacy issues you're carrying over.
> It's pretty much always best to rebuild it from scratch to a known state and
> re-add it.  Automated build processes are worth their weight in gold for
> these circumstances.
> Regards,
> Paul Angus
> Cloud Architect
> S: +44 20 3603 0540 | M: +447711418784 | T: @CloudyAngus
> -----Original Message-----
> From: Garith Dugmore []
> Sent: 22 October 2014 15:54
> To:
> Subject: HA issue and Xen resets
> Hi All,
> I'm new to cloudstack and busy testing out ACS 4.3.1 on Centos 6.4 using
> Xenserver 6.2. I have the management server setup and have 2 xen servers
> that I'm testing out at the moment; specifically the HA functionality.
> After getting an instance up and running I yanked the network cable out of
> the one xen server and awaited the HA awesomeness to kick in. Both hosts
> still remained in "Up" state even though the one was no longer pingable and
> the instance that was hosted on that xen host still showed "Running" even
> though I also couldn't ping it. After some reading one suggestion was
> setting 'alert.wait' to 30 and restarting cloudstack-management. After that
> didn't seem to do anything after waiting a while I rebooted the management
> server all together and found that both hosts were marked as disconnected.
> I've tried going in and out of maintenance mode and ended up deleting the one
> xen host that was still reachable thinking I could just re-add it but I
> received an error in doing so. I have read somewhere that once you've either
> reinstalled the management server or removed a xen host you need to
> re-install the xen host. Is this true? I was hoping for a factory reset
> command of some sort.
> Besides my obvious HA problems and host disconnect issues which I'd love some
> pointers on are there any pointers on Xen server resets?
> Note I have attempted a 4.4.0 install on centos and after a couple issues
> that I can no longer recall I ended up with a way easier install on 4.3.1
> which is why I've stuck with it for now.
> Any pointers will be greatly appreciated. Willing to try anything!
> --
> Garith Dugmore
> South African Astronomical Observatory
> and Southern African Large Telescope
> Find out more about ShapeBlue and our range of CloudStack related services
> IaaS Cloud Design & Build<>
> CSForge – rapid IaaS deployment framework<>
> CloudStack Consulting<>
> CloudStack Infrastructure
> Support<>
> CloudStack Bootcamp Training
> Courses<>
> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender
> if you believe you have received this email in error. Shape Blue Ltd is a
> company incorporated in England & Wales. ShapeBlue Services India LLP is a
> company incorporated in India and is operated under license from Shape Blue
> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
> and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a
> company registered by The Republic of South Africa and is traded under
> license from Shape Blue Ltd. ShapeBlue is a registered trademark.
> !DSPAM:5447d06316022015084578!


Garith Dugmore 
Senior System Administrator 
South African Astronomical Observatory 
and Southern African Large Telescope 

SAAO Telephone: +27 21 447 0025 
Direct Line: +27 21 460 6296 
Ext: 4004 
SAAO Website: 
SALT Website: 
Skype: garith5512 

View raw message