www-builds mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lance Albertson <la...@osuosl.org>
Subject [Hosting] Ganeti Internal Node Rebuild - Nov 30, 2017 3:00pm PST (Nov 30 2300 UTC)
Date Tue, 28 Nov 2017 21:20:09 GMT
Service(s) affected:

The following hosts will need to be non-live migrated to their secondary
nodes (i.e. shutdown and start is required):

ash.osuosl.org (lists.osuosl.org, OSL mail store, OSL shell server)
ds2.osuosl.org (ldap2.osuosl.org)
hemlock.osuosl.org (smtp2.osuosl.org)
internal-web1.osuosl.org
mon2.osuosl.org (munin)
openpower-controller.osuosl.org (OpenPOWER OpenStack controller)
oprod-controller1.osuosl.org (OSL OpenStack controller)

We expect the outages for each host to be short (under 5 minutes each)
since they are VMs.

Outage Window:
Start:   Thu, Nov 30, 3:00PM PST (Thu Nov 30 2300 UTC)
End:     Thu, Nov 30, 5:00PM PST (Fri Dec 1 0100 UTC)

Reason for outage:

We're in the midst of rebuilding our Ganeti clusters to CentOS 7 and
managed via Chef. The first step will be to rebuild our internal Ganeti
cluster which hosts some OSL specific hosts, some of which are running
shared services such as Mailman and LDAP. This specific outage will free up
ginternal2 so that it can be rebuilt and added back as a Ganeti node. All
secondary instances attached to this node will remain and be re-synced once
the node has been rebuilt and added back as a node. All VM data stored on
this node will remain intact during the rebuild as only the OS partition
will be rebuilt.

We're also going to take advantage of this downtime to replace the RAID
batteries on this node. We'll be doing the same for the last node in this
cluster later next week depending on how this rebuild goes and will send
another email out for that outage.

If you have any questions or concerns please let us know.

Projects affected:

None specifically

-- 
Lance Albertson
Director
Oregon State University | Open Source Lab

Mime
View raw message