incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chiradeep Vittal <Chiradeep.Vit...@citrix.com>
Subject Re: Issues when vCenter becomes unavailable
Date Fri, 22 Feb 2013 20:41:28 GMT
CC'ing Kelven to see if he has any ideas.

On 2/22/13 12:22 PM, "Dave Dunaway" <dave.dunaway@gmail.com> wrote:

>If I may suggest also testing a disconnect of a host (hypervisor) from
>vcenter, so that vcenter and CS can still talk, but vcenter cannot talk to
>the hosts (hypervisors). CS marks the host as down or failed or whatever.
>
>When the host comes back up vcenter can it just fine and all seems good.
>That however is not the case (I had this with CS 3.0.5 and vmware esxi
>5.0)
>when CS tries to talk to vcenter and the previously disconnected host
>(that
>is now recovered).
>
>What we experienced was that we had to migrate all guests off the
>recovered
>host, and then destroy that host in CS, and re-create it. Then we could
>migrate back onto it the guests which had been previously migrated.
>
>The curious thing is that while CS did not want to send commands to the
>host (it kept on saying host id=X has timedout when whatever command was
>sent to it), CS WAS polling the host for resources and getting the correct
>numbers.... so CS could in some ways talk to the host (ie: it knew the
>capabilities, number of VMs on it, etc).
>
>Luckily for me this all happened in a test environment. In production,
>this
>would have been a real nightmare!
>
>
>dave
>
>
>On Fri, Feb 22, 2013 at 2:48 PM, Musayev, Ilya <imusayev@webmd.net> wrote:
>
>> Andi
>>
>> I'm on CS4.0. I simulated the VMWare VCenter 5 failure by adding a bogus
>> IP entry in /etc/hosts for 10 minutes for virtual center host. That in
>>turn
>> made VC unreachable by CS.
>>
>> I then began executing commands and sure enough commands failed or
>> backlogged. Once I restored VC connectivity, the backlogged commands
>> executed and I did not experience any abnormalities.
>>
>> I will redo this test and leave VC off for an hour - maybe a need a
>>longer
>> outage.
>>
>> Regards
>> ilya
>>
>>
>>
>> -----Original Message-----
>> From: Musayev, Ilya
>> Sent: Thursday, February 21, 2013 2:43 PM
>> To: cloudstack-users@incubator.apache.org
>> Subject: RE: Issues when vCenter becomes unavailable
>>
>> This is definitely not the behavior we want with vcenter.
>>
>> I will test this out on my lab setup shortly.
>>
>> Thanks
>> ilya
>>
>> -----Original Message-----
>> From: Chip Childers [mailto:chip.childers@sungard.com]
>> Sent: Thursday, February 21, 2013 9:40 AM
>> To: cloudstack-users@incubator.apache.org
>> Subject: Re: Issues when vCenter becomes unavailable
>>
>> On Thu, Feb 21, 2013 at 08:59:14AM -0500, Mathias Mullins wrote:
>> > Andreas,
>> >
>> > The open source community doesn't support the Citrix version 3.0.6.
>> > You need to report this via your Citrix Support contract. Sounds like
>> > this could be a bug.
>> >
>> > Community - this could be a possible issue in 4.0.0 / 4.0.1. I don't
>> > know if this test case has been explored.
>>
>> Thx - I forwarded to cs-dev@i.a.o to get the test engineers in the
>> community to take a look.
>>
>> >
>> > Thanks,
>> > Matt Mullins
>> > CloudPlatform Implementation Engineer
>> > Worldwide Cloud Services  Citrix System, Inc.
>> > +1 (407) 920-1107  Office/Cell Phone
>> > matt.mullins@citrix.com
>> >
>> >
>> >
>> > On 2/21/13 5:35 AM, "Fuchs, Andreas (SwissTXT)"
>> > <Andreas.Fuchs@swisstxt.ch> wrote:
>> >
>> > >Hi CS Users
>> > >
>> > >We are running CS 3.0.6 on a vSphere platform and found a strange
>> > >behavior.
>> > >
>> > >When the vCenter becomes unavailable due to a reboot or some other
>> > >issue, it seems that CS is shutting down instances when vCenter
>> > >becomes available again.
>> > >
>> > >What we think what happens.
>> > >1. vCenter becomes unrechabale
>> > >2. CS marks the ESX servers as "down"
>> > >3. We think this leads to: CS marks the instances as down as well 4.
>> > >When vCenter becomes available again, CS stops the "marked as down"
>> > >instances
>> > >
>> > >This is very bad as the Instances where running all the time and the
>> > >the shutdown issued by CS is forcing a service interruption.
>> > >
>> > >My problem is that I cannot realy reporoduce as allot of testing is
>> > >ongoing on the platform at the moment, so my question:
>> > >
>> > >Does someone else see this issue as well and can maybe reproduce?
>> > >Is there a workaround to it, can I change some flag or something
>> > >which tells CS to never shut down an instance by himself?
>> > >Why are the ESX hosts getting marked as down and not unreachable or
>> > >something?
>> > >
>> > >Best regards
>> > >Andi
>> >
>> >
>>
>>
>>


Mime
View raw message