Return-Path: X-Original-To: apmail-cloudstack-issues-archive@www.apache.org Delivered-To: apmail-cloudstack-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 45C7810FDA for ; Tue, 19 Nov 2013 22:07:21 +0000 (UTC) Received: (qmail 90514 invoked by uid 500); 19 Nov 2013 22:07:21 -0000 Delivered-To: apmail-cloudstack-issues-archive@cloudstack.apache.org Received: (qmail 90413 invoked by uid 500); 19 Nov 2013 22:07:21 -0000 Mailing-List: contact issues-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cloudstack.apache.org Delivered-To: mailing list issues@cloudstack.apache.org Received: (qmail 90404 invoked by uid 500); 19 Nov 2013 22:07:21 -0000 Delivered-To: apmail-incubator-cloudstack-issues@incubator.apache.org Received: (qmail 90388 invoked by uid 99); 19 Nov 2013 22:07:21 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 19 Nov 2013 22:07:21 +0000 Date: Tue, 19 Nov 2013 22:07:20 +0000 (UTC) From: "Anthony Xu (JIRA)" To: cloudstack-issues@incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Assigned] (CLOUDSTACK-2140) Host is still marked as being in "Up" state when the host is shutdown (when there are no more hosts in the cluster) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CLOUDSTACK-2140?page=3Dcom.atl= assian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anthony Xu reassigned CLOUDSTACK-2140: -------------------------------------- Assignee: (was: Anthony Xu) > Host is still marked as being in "Up" state when the host is shutdown (wh= en there are no more hosts in the cluster) > -------------------------------------------------------------------------= ------------------------------------------ > > Key: CLOUDSTACK-2140 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-214= 0 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the defa= ult.)=20 > Components: Management Server > Affects Versions: 4.2.0, 4.2.1 > Environment: build from master > Reporter: Sangeetha Hariharan > Priority: Critical > Fix For: Future > > Attachments: management-server.rar > > > Host is still marked as being in "Up" state when the host is shutdown (wh= en there are no more hosts in the cluster. > Set up: > Advanced zone. > 3 hosts in a cluster ( in my case host id - 7 ,8 ,9 ). > I did not have any problems when host 8 and host 9 where shutdown. > When I tried to shutdown host 7 , I see the host still being in "Up" stat= e , even after the management server detected that it is not able to connec= t with this host. > Following exception seen in management server logs: > 2013-04-22 14:48:18,350 DEBUG [xen.resource.XenServerConnectionPool] (Dir= ectAgent-350:null) localLogout has problem Failed to read server's response= : connect timed out > 2013-04-22 14:48:18,350 WARN [xen.resource.CitrixResourceBase] (DirectAg= ent-350:null) Unable to stop i-3-45-VM due to > com.cloud.utils.exception.CloudRuntimeException: Unable to reset master o= f slave 10.223.59.4 to 10.223.59.2 due to org.apache.xmlrpc.XmlRpcException= : Failed to read server's response: connect timed out > at com.cloud.hypervisor.xen.resource.XenServerConnectionPool.Pool= EmergencyResetMaster(XenServerConnectionPool.java:443) > at com.cloud.hypervisor.xen.resource.XenServerConnectionPool.conn= ect(XenServerConnectionPool.java:661) > at com.cloud.hypervisor.xen.resource.CitrixResourceBase.getConnec= tion(CitrixResourceBase.java:5583) > at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(C= itrixResourceBase.java:3728) > at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRe= quest(CitrixResourceBase.java:474) > at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeR= equest(XenServer56Resource.java:73) > at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgen= tAttache.java:186) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.= java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:= 334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutu= reTask.access$101(ScheduledThreadPoolExecutor.java:165) > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutu= reTask.run(ScheduledThreadPoolExecutor.java:266) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolEx= ecutor.java:1110) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolE= xecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2013-04-22 14:48:18,364 DEBUG [agent.manager.DirectAgentAttache] (DirectA= gent-350:null) Seq 9-72160431: Response Received: > 2013-04-22 14:48:18,370 DEBUG [agent.transport.Request] (DirectAgent-350:= null) Seq 9-72160431: Processing: { Ans: , MgmtId: 7508777239729, via: 9, = Ver: v1, Flags: 110, [{"StopAnswer":{"result":false,"details":"Exception: c= om.cloud.utils.exception.CloudRuntimeException\nMessage: Unable to reset ma= ster of slave 10.223.59.4 to 10.223.59.2 due to org.apache.xmlrpc.XmlRpcExc= eption: Failed to read server's response: connect timed out\nStack: com.clo= ud.utils.exception.CloudRuntimeException: Unable to reset master of slave 1= 0.223.59.4 to 10.223.59.2 due to org.apache.xmlrpc.XmlRpcException: Failed = to read server's response: connect timed out\n\tat com.cloud.hypervisor.xen= .resource.XenServerConnectionPool.PoolEmergencyResetMaster(XenServerConnect= ionPool.java:443)\n\tat com.cloud.hypervisor.xen.resource.XenServerConnecti= onPool.connect(XenServerConnectionPool.java:661)\n\tat com.cloud.hypervisor= .xen.resource.CitrixResourceBase.getConnection(CitrixResourceBase.java:5583= )\n\tat com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(Citrix= ResourceBase.java:3728)\n\tat com.cloud.hypervisor.xen.resource.CitrixResou= rceBase.executeRequest(CitrixResourceBase.java:474)\n\tat com.cloud.hypervi= sor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.jav= a:73)\n\tat com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgent= Attache.java:186)\n\tat java.util.concurrent.Executors$RunnableAdapter.call= (Executors.java:471)\n\tat java.util.concurrent.FutureTask$Sync.innerRun(Fu= tureTask.java:334)\n\tat java.util.concurrent.FutureTask.run(FutureTask.jav= a:166)\n\tat java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutu= reTask.access$101(ScheduledThreadPoolExecutor.java:165)\n\tat java.util.con= current.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThread= PoolExecutor.java:266)\n\tat java.util.concurrent.ThreadPoolExecutor.runWor= ker(ThreadPoolExecutor.java:1110)\n\tat java.util.concurrent.ThreadPoolExec= utor$Worker.run(ThreadPoolExecutor.java:603)\n\tat java.lang.Thread.run(Thr= ead.java:679)\n","wait":0}}] } > 2013-04-22 14:48:18,370 DEBUG [agent.transport.Request] (DirectAgent-276:= null) Seq 9-72160431: Received: { Ans: , MgmtId: 7508777239729, via: 9, Ve= r: v1, Flags: 110, { StopAnswer } } > 2013-04-22 14:48:18,370 WARN [cloud.vm.VirtualMachineManagerImpl] (Direc= tAgent-276:null) Unable to actually stop VM[User|anan5] but continue with r= elease because it's a force stop > 2013-04-22 14:48:18,371 WARN [agent.manager.DirectAgentAttache] (DirectA= gent-276:null) Seq 7-1177944069: Exception caught > com.cloud.utils.exception.CloudRuntimeException: Unable to stop the virtu= al machine due to Exception: com.cloud.utils.exception.CloudRuntimeExceptio= n > Host entries in DB: > | 7 | Rack3Host17.lab.vmops.com | ea8= a5618-3e10-4a02-a6ea-9a8e10e7efc7 | Up | Routing | 10.223.59= .2 | 255.255.255.192 | bc:30:5b:d4:1c:36 | 10.223.59.2 | 25= 5.255.255.192 | bc:30:5b:d4:1c:36 | NULL | NULL = | NULL | 6 | 10.223.59.2 | 255.255.255.= 192 | bc:30:5b:d4:1c:36 | NULL | 1 | 4 | 4 | 2= 261 | iqn.2005-03.org.open-iscsi:7ad5ccd9c587 | NULL = | XenServer | 6.0.2 | 16190149248 | com.cloud.hypervis= or.xen.resource.XenServer602Resource | 4.2.0-SNAPSHOT | NULL = | NULL | xen-3.0-x86_64 , xen-3.0-x86_32p , hvm-3= .0-x86_32 , hvm-3.0-x86_32p , hvm-3.0-x86_64 | fb549fb6-82b6-4b74-a468-c511= d744b238 | 1 | 1 | 0 | 133430= 7227 | 7508777239729 | 2013-04-19 00:20:22 | 2013-04-18 21:22:16 | NULL = | 6 | Enabled | NULL | NULL | Disabled | > | 8 | Rack3Host18.lab.vmops.com | df1= afd25-4871-41f7-bccb-44d8f9ce193d | Down | Routing | 10.223.59= .3 | 255.255.255.192 | bc:30:5b:d4:23:54 | 10.223.59.3 | 25= 5.255.255.192 | bc:30:5b:d4:23:54 | NULL | NULL = | NULL | 6 | 10.223.59.3 | 255.255.255.= 192 | bc:30:5b:d4:23:54 | NULL | 1 | 4 | 4 | 2= 261 | iqn.2005-03.org.open-iscsi:8191f9f922ef | NULL = | XenServer | 6.0.2 | 16190149248 | com.cloud.hypervis= or.xen.resource.XenServer602Resource | 4.2.0-SNAPSHOT | NULL = | NULL | xen-3.0-x86_64 , xen-3.0-x86_32p , hvm-3= .0-x86_32 , hvm-3.0-x86_32p , hvm-3.0-x86_64 | b127d031-d3bc-4859-bc52-6339= 62ba61a9 | 1 | 1 | 0 | 133463= 5374 | NULL | 2013-04-19 00:20:22 | 2013-04-18 21:23:10 | NULL = | 90 | Enabled | NULL | NULL | Disabled | > | 9 | Rack3Host19.lab.vmops.com | 565= dff5f-e17a-4216-a20a-6283e2bab0bf | Down | Routing | 10.223.59= .4 | 255.255.255.192 | bc:30:5b:d4:15:d2 | 10.223.59.4 | 25= 5.255.255.192 | bc:30:5b:d4:15:d2 | NULL | NULL = | NULL | 6 | 10.223.59.4 | 255.255.255.= 192 | bc:30:5b:d4:15:d2 | NULL | 1 | 4 | 4 | 2= 261 | iqn.2005-03.org.open-iscsi:f3aa69c5a08c | NULL = | XenServer | 6.0.2 | 3701658240 | com.cloud.hypervis= or.xen.resource.XenServer602Resource | 4.2.0-SNAPSHOT | NULL = | NULL | xen-3.0-x86_64 , xen-3.0-x86_32p , hvm-3= .0-x86_32 , hvm-3.0-x86_32p , hvm-3.0-x86_64 | bae94938-b22d-4874-8f18-1e96= 15338b3c | 1 | 1 | 0 | 133463= 7141 | NULL | 2013-04-19 00:20:22 | 2013-04-18 21:23:22 | NULL = | 67 | Enabled =20 -- This message was sent by Atlassian JIRA (v6.1#6144)