cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Mabry <dma...@ena.com>
Subject Re: [DISCUSS][FS] Host HA for CloudStack
Date Fri, 17 Feb 2017 22:10:25 GMT

On 2/16/17, 5:18 AM, "Rohit Yadav" <rohit.yadav@shapeblue.com> wrote:

    All,
    
    
    I would like to start discussion on a new feature - Host HA for CloudStack.
    
    CloudStack lacks a way to reliably fence a host, the idea of the host-ha feature is to
provide a general purpose HA framework and HA provider implementation specific for hypervisor
that can use additional mechanism such as OOBM (ipmi based power management) to reliably investigate,
recover and fence a host. This feature can handle scenarios associated with server crash issues
and reliable fencing of hosts and HA of VM. The first version will have HA provider implementation
for KVM (and for simulator to test the framework implementation, and write marvin tests that
can validate the feature on Travis and others).
    
    
    Please have a look at the FS here:
    
    https://cwiki.apache.org/confluence/display/CLOUDSTACK/Host+HA
    
    
    Looking forward to your comments and questions.
    
    
    Regards.
    
    rohit.yadav@shapeblue.com 
    www.shapeblue.com
    53 Chandos Place, Covent Garden, London  WC2N 4HSUK
    @shapeblue
      
     
    
 Rohit,

First, thanks for all the work you have put into this.  This is something that CS has sorely
needed for a long time.

A couple of items:

1.) You state the following:
“Before invoking the HA provider’s fence operation, the HA resource management will place
the resource in maintenance mode. The intention is to require an administrator to manually
verify that a resource is ready to return service by requiring an administrator to take it
out of maintenance mode.”
I agree that putting a host in maintenance mode to require manual intervention in order to
bring it back online is ideal and honestly how I would probably prefer to do it.  However,
I also like to give the end user/operator choice.  Perhaps we could add an option to bring
the Host out of Maintenance mode automatically if it passes all checks and comes back into
an ELIGIBLE state.  This way, if the operator chooses, the host could come back into full
operation and start recovering VMs if needed.  This could also be handy if your environment
isn’t quite n+1 when it comes to host capacity and you need to have the host back up and
running as soon as possible to minimize the outage duration.  Again, I know it isn’t ideal,
but I don’t see the harm in giving the operator the choice.

2.) You state the following:
“For the initial release, only KVM with NFS storage will be supported. However, the storage
check component will be implemented in a modular fashion allowing for checks using other storage
platforms(e.g. Ceph) in the future. HA provider plugins can be implemented for other hypervisors.”
We are using KVM with a Ceph backend and would be very interested in helping make it a part
of the initial push for this feature.  I have a Dev environment backed by Ceph that we could
use for teseting and would be willing to help with the development of the Ceph activity checks.

I’m looking forward to getting this feature added to CS.  Again, great job putting this
together and starting the conversation.

Thanks,
Mabry

Mime
View raw message