cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Makrand <makrandsa...@gmail.com>
Subject Upgrade of Primary storage.
Date Fri, 13 May 2016 06:58:28 GMT
Guys,

Need little help. So in my second week on new job, I am suppose to upgrade
Nexeta (Primary storage and secondary storage for cloud) OS.

Overview of setup:-

1Zone>>1POD>>1 Cluster>>8 Hosts

Hypervisor:- Citrix Xenserer 6.5 (Free edition)

Storage Primary (Cluster Level):- On nexenta

Secondary Storage:- On same nexenta BOX (which caters primary)

Management Node is VM (Ubuntu 12004 LTS) whos root disk resides again on
primary storage coming from same nexenta BOX. Seems LUN is different. This
VM is running on Management cluster (2 XEN 6.5 hosts).


Well not sure why and how they kept single storage BOX for all primary and
secondary. Pretty wiered. It was done by a Vendor or someone long back.
Thats whole different story.

Any-who, coming to point of this email. I've chalked out plan like below.
------------------------------------------------------
A) Shutdown all VMs in following order:- First user VMs, then VRs & then
System VMs.

B) Put all XEN hosts in MM (Maintenance mode) from cloudstack.

 1) Verify that all hosts are down (Shutdown them one by one) from XEN
Center

C) ssh into Management Server VM.  Backup cloudstack DB save it on JumpBOX
(Use WinSCP)

D) Shutdown Cloudstack Management Server VM and other VMs on management
cluster. This needs to be done by issuing 'shutdown' command from OS level.

E) Shutdown Management cluster XEN hosts.

F) Upgrade the storage.  (Reboot is needed at end.......hence so many
shutdowns)

G) Once upgrade is done start the Machines (Physical and Virtual) in
reveres order than that of shutdown. (Step E to A above) Something like
below

 1) Management clsuter XEN hosts
 2) Cloudstack management server VM and other management cluster VMs
 3) XEN Hypervisor Nodes under cloud.
 4) Take XEN hosts out of MM from cloudstack admin GUI.
 5) Start SSVM and console proxy VM (I guess this time cloudstack will
recreate CPVM and SSVM if we      start first user VM.....not sure if this
step will be needed)
 6) Customer VMs and corresponding VRs

H) Verify from and try to deploy test VM.
------------------------------------------------------

All this downtime is ok. Do you think I am missing anything. Any comments
on improvising? Should I expect some more glitches? What was your previous
experience with primary storage upgrade.

Note:-
Last time same upgrade happened for one of similar zone and there was issue
of mapping primary storage LUNs with XEN hosts. Restart of hosts did trick
that time and hence my manager wants to shutdown all physical hosts this
time (Since all VMs have their disks on this storage box...which will
reboot once or twice during upgrade)

BTW

1) Is it necessary to put XEN hosts in MM from cloudstack? (If I am
shutting down all VMs).

2) After I bring UP Management server in step G-1, will it still have host
in MM? or What would it try to do after its up? (Assuming all hosts in
cluster are down at that time.

Thanks for reading.

--
Best,
Makrand

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message