cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Koushik Das <koushik....@citrix.com>
Subject RE: cs 4.1 host disconnected status
Date Fri, 12 Jul 2013 08:17:53 GMT
Looks like the KVM investigator is not able to determine the state of the agent. Can you share
the full log?

> -----Original Message-----
> From: Valery Ciareszka [mailto:valery.tereshko@gmail.com]
> Sent: Thursday, July 11, 2013 7:47 PM
> To: users
> Subject: cs 4.1 host disconnected status
> 
> Hi all.
> 
> I use the following environment: CS 4.1, KVM, Centos 6.4
> (management+node1+node2), OpenIndiana NFS server as primary and
> secondary storage.
> and I have the following problem:
> If I switch one hypervisor node off via ipmi (simulate server crash), it never
> goes to Disconnected status in management. Accordingly, ha-enabled VMs
> are not restarted on another hypervisor node, because it believes that
> disconnected node is still online.
> 
> 
> I get following in management server logs:
> 
> 2013-07-11 10:19:16,153 DEBUG [agent.transport.Request]
> (AgentManager-Handler-13:null) Seq 19-1133189098:             Processing:
>  { Ans: , MgmtId: 161603152803976, via: 19, Ver: v1, Flags: 10,
> [{"Answer":{"result":false,"details":     "Unable to ping computing host,
> exiting","wait":0}}] }
> 2013-07-11 10:19:16,153 DEBUG [agent.transport.Request]
> (AgentTaskPool-1:null) Seq 19-1133189098: Received:  { Ans: , MgmtId:
> 161603152803976, via: 19, Ver: v1, Flags: 10, { Answer } }
> 2013-07-11 10:19:16,153 DEBUG [cloud.ha.AbstractInvestigatorImpl]
> (AgentTaskPool-1:null) host (172.16.20.241) cannot  be pinged, returning null
> ('I don't know')
> 2013-07-11 10:19:16,153 DEBUG [cloud.ha.UserVmDomRInvestigator]
> (AgentTaskPool-1:null) could not reach agent, could   not reach agent's
> host, returning that we don't have enough information
> 2013-07-11 10:19:16,153 DEBUG [cloud.ha.HighAvailabilityManagerImpl]
> (AgentTaskPool-1:null) null unable to determine  the state of the host.
>  Moving on.
> 2013-07-11 10:19:16,153 DEBUG [cloud.ha.HighAvailabilityManagerImpl]
> (AgentTaskPool-1:null) null unable to determine  the state of the host.
>  Moving on.
> 2013-07-11 10:19:16,153 WARN  [agent.manager.AgentManagerImpl]
> (AgentTaskPool-1:null) Agent state cannot be           determined, do
> nothing
> 
> 
> If I power on dead node, it goes to state "Connecting" and then "Up" in
> management interface.
> 
> 2013-07-11 13:57:24,311 DEBUG [cloud.host.Status] (Thread-6:null) Ping
> timeout for host 12, do invstigation
> 2013-07-11 13:58:24,315 DEBUG [cloud.host.Status] (Thread-6:null) Ping
> timeout for host 12, do invstigation
> 2013-07-11 13:59:24,320 DEBUG [cloud.host.Status] (Thread-6:null) Ping
> timeout for host 12, do invstigation
> 2013-07-11 13:59:57,239 DEBUG [cloud.host.Status]
> (AgentConnectTaskPool-5:null) Transition:[Resource state = Enabled, Agent
> event = AgentConnected, Host id = 12, name = ad112.colobridge.net]
> 2013-07-11 13:59:57,264 DEBUG [cloud.host.Status]
> (AgentConnectTaskPool-5:null) Agent status update: [id = 12; name =
> ad112.colobridge.net; old status = Up; event = AgentConnected; new status
> = Connecting; old update count = 1285; new update count = 1286]
> 2013-07-11 14:00:50,611 DEBUG [cloud.host.Status]
> (AgentConnectTaskPool-5:null) Transition:[Resource state = Enabled, Agent
> event = Ready, Host id = 12, name = ad112.colobridge.net]
> 2013-07-11 14:00:50,633 DEBUG [cloud.host.Status]
> (AgentConnectTaskPool-5:null) Agent status update: [id = 12; name =
> ad112.colobridge.net; old status = Connecting; event = Ready; new status =
> Up; old update count = 1286; new update count = 1287]
> 
> 
> If I restart cloud-management service, dead node goes to state
> "Disconnected" in management interface.
> (there is nothing special in logs in this case)
> 
> If I do nothing,  dead node could stay in "Up" state forever (I waited for
> 12 hours) in management interface, throwing into logs "Agent state cannot
> be determined, do nothing"
> 
> Would appreciate if someone could help/suggest how to deal with this
> problem.
> 
> --
> Regards,
> Valery
> 
> http://protocol.by/slayer
Mime
View raw message