Return-Path: X-Original-To: apmail-cloudstack-users-archive@www.apache.org Delivered-To: apmail-cloudstack-users-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3375118A87 for ; Mon, 6 Jul 2015 08:42:31 +0000 (UTC) Received: (qmail 11094 invoked by uid 500); 6 Jul 2015 08:42:25 -0000 Delivered-To: apmail-cloudstack-users-archive@cloudstack.apache.org Received: (qmail 11042 invoked by uid 500); 6 Jul 2015 08:42:25 -0000 Mailing-List: contact users-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@cloudstack.apache.org Delivered-To: mailing list users@cloudstack.apache.org Received: (qmail 11031 invoked by uid 99); 6 Jul 2015 08:42:25 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 06 Jul 2015 08:42:25 +0000 X-ASF-Spam-Status: No, hits=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of prvs=622356e7d=koushik.das@citrix.com designates 103.14.252.240 as permitted sender) Received: from [103.14.252.240] (HELO SMTP.CITRIX.COM.AU) (103.14.252.240) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 06 Jul 2015 08:40:11 +0000 X-IronPort-AV: E=Sophos;i="5.15,413,1432598400"; d="scan'208";a="18889103" From: Koushik Das To: "users@cloudstack.apache.org" Subject: RE: NPE while adding host into maintenance in ACS 4.5.1 Thread-Topic: NPE while adding host into maintenance in ACS 4.5.1 Thread-Index: AdC3wuC+3pYgCSrKTfGmUIPJEpDGvQABI1Qg Date: Mon, 6 Jul 2015 08:41:53 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-DLP: SIN1 X-Virus-Checked: Checked by ClamAV on apache.org This should fix the problem https://issues.apache.org/jira/browse/CLOUDSTAC= K-8570. Check if the fix is present in the build you are using. -----Original Message----- From: Sonali Jadhav [mailto:sonali@servercentralen.se]=20 Sent: Monday, 6 July 2015 13:38 To: users@cloudstack.apache.org Subject: NPE while adding host into maintenance in ACS 4.5.1 Hi, In order to update xenserver from 6.2 to 6.5, I added pool master in mainte= nance mode from ACS. In ACS it said successful , and it started to move vms= to another host in cluster, except few since xenserver tool was not instal= led on them. So I shut those vms down from acs. But host was not sent to ma= intenance in xencenter. My host went to "alert" state. And now I can't do a= ny operations since it's in alert state. I am seeing this error in logs, 015-07-06 09:54:02,400 DEBUG [c.c.a.t.Request] (AgentTaskPool-1:ctx-a468547= 3) Seq 1-1601874092460343301: Received: { Ans: , MgmtId: 59778234354585, v= ia: 1, Ver: v1, Flags: 10, { ModifyStoragePoolAnswer } } 2015-07-06 09:54:02,415 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-165= :ctx-fe175294) Seq 5-8999880905346524321: Executing request 2015-07-06 09:54:02,437 INFO [o.a.c.s.d.p.DefaultHostListener] (AgentTaskP= ool-1:ctx-a4685473) Connection established between storage pool org.apache.= cloudstack.storage.datastore.PrimaryDataStoreImpl@74be27a7 and host 1 2015-07-06 09:54:02,444 DEBUG [c.c.s.StorageManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Found storage pool PrimStore of type NetworkFilesystem with ov= erprovisioning factor 2 2015-07-06 09:54:02,445 DEBUG [c.c.s.StorageManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Total over provisioned capacity calculated is 2 * 115794950225= 92 2015-07-06 09:54:02,445 DEBUG [c.c.s.StorageManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Total over provisioned capacity of the pool PrimStore id: 1 is= 23158990045184 2015-07-06 09:54:02,446 DEBUG [c.c.s.StorageManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Successfully set Capacity - 23158990045184 for capacity type -= 3 , DataCenterId - 1, HostOrPoolId - 1, PodId 1 2015-07-06 09:54:02,446 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Connect to listener: DeploymentPlanningManagerImpl 2015-07-06 09:54:02,446 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Connect to listener: StorageCapacityListener 2015-07-06 09:54:02,447 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Connect to listener: ComputeCapacityListener 2015-07-06 09:54:02,451 DEBUG [c.c.c.CapacityManagerImpl] (AgentTaskPool-1:= ctx-a4685473) Found 0 VMs on host 1 2015-07-06 09:54:02,458 DEBUG [c.c.c.CapacityManagerImpl] (AgentTaskPool-1:= ctx-a4685473) Found 4 VM, not running on host 1 2015-07-06 09:54:02,459 ERROR [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Monitor ComputeCapacityListener says there is an error in the = connect process for 1 due to null java.lang.NullPointerException at com.cloud.capacity.CapacityManagerImpl.updateCapacityFor= Host(CapacityManagerImpl.java:648) at com.cloud.capacity.ComputeCapacityListener.processConnec= t(ComputeCapacityListener.java:66) at com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsO= fConnection(AgentManagerImpl.java:539) at com.cloud.agent.manager.AgentManagerImpl.handleDirectCon= nectAgent(AgentManagerImpl.java:1447) at com.cloud.resource.ResourceManagerImpl.createHostAndAgen= t(ResourceManagerImpl.java:1794) at com.cloud.resource.ResourceManagerImpl.createHostAndAgen= t(ResourceManagerImpl.java:1920) at sun.reflect.GeneratedMethodAccessor263.invoke(Unknown So= urce) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Delegati= ngMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.springframework.aop.support.AopUtils.invokeJoinpoint= UsingReflection(AopUtils.java:317) at org.springframework.aop.framework.ReflectiveMethodInvoca= tion.invokeJoinpoint(ReflectiveMethodInvocation.java:183) at org.springframework.aop.framework.ReflectiveMethodInvoca= tion.proceed(ReflectiveMethodInvocation.java:150) at org.springframework.aop.interceptor.ExposeInvocationInte= rceptor.invoke(ExposeInvocationInterceptor.java:91) at org.springframework.aop.framework.ReflectiveMethodInvoca= tion.proceed(ReflectiveMethodInvocation.java:172) at org.springframework.aop.framework.JdkDynamicAopProxy.inv= oke(JdkDynamicAopProxy.java:204) at com.sun.proxy.$Proxy149.createHostAndAgent(Unknown Sourc= e) at com.cloud.agent.manager.AgentManagerImpl$SimulateStartTa= sk.runInContext(AgentManagerImpl.java:1078) at org.apache.cloudstack.managed.context.ManagedContextRunn= able$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManage= dContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManage= dContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManage= dContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunn= able.run(ManagedContextRunnable.java:46) at java.util.concurrent.ThreadPoolExecutor.runWorker(Thread= PoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Threa= dPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) 2015-07-06 09:54:02,460 INFO [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Host 1 is disconnecting with event AgentDisconnected 2015-07-06 09:54:02,462 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) The next status of agent 1is Alert, current status is Connecti= ng 2015-07-06 09:54:02,462 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Deregistering link for 1 with state Alert 2015-07-06 09:54:02,462 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Remove Agent : 1 2015-07-06 09:54:02,462 DEBUG [c.c.a.m.DirectAgentAttache] (AgentTaskPool-1= :ctx-a4685473) Processing disconnect 1(SeSolXS01) 2015-07-06 09:54:02,462 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Disconnect to listener: com.cloud.hypervisor.xenserver= .discoverer.XcpServerDiscoverer 2015-07-06 09:54:02,462 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Disconnect to listener: com.cloud.hypervisor.hyperv.di= scoverer.HypervServerDiscoverer 2015-07-06 09:54:02,462 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Disconnect to listener: org.apache.cloudstack.engine.o= rchestration.NetworkOrchestrator 2015-07-06 09:54:02,462 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Disconnect to listener: com.cloud.hypervisor.vmware.ma= nager.VmwareManagerImpl 2015-07-06 09:54:02,462 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Disconnect to listener: com.cloud.network.security.Sec= urityGroupListener 2015-07-06 09:54:02,462 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Disconnect to listener: com.cloud.vm.ClusteredVirtualM= achineManagerImpl 2015-07-06 09:54:02,462 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Disconnect to listener: com.cloud.storage.secondary.Se= condaryStorageListener 2015-07-06 09:54:02,462 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Disconnect to listener: com.cloud.storage.listener.Sto= ragePoolMonitor 2015-07-06 09:54:02,462 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Disconnect to listener: com.cloud.deploy.DeploymentPla= nningManagerImpl 2015-07-06 09:54:02,462 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Disconnect to listener: com.cloud.capacity.StorageCapa= cityListener 2015-07-06 09:54:02,462 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Disconnect to listener: com.cloud.capacity.ComputeCapa= cityListener 2015-07-06 09:54:02,463 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Disconnect to listener: com.cloud.storage.LocalStorage= PoolListener 2015-07-06 09:54:02,463 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Disconnect to listener: com.cloud.network.NetworkUsage= ManagerImpl$DirectNetworkStatsListener 2015-07-06 09:54:02,463 DEBUG [c.c.n.NetworkUsageManagerImpl] (AgentTaskPoo= l-1:ctx-a4685473) Disconnected called on 1 with status Alert 2015-07-06 09:54:02,463 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Disconnect to listener: com.cloud.network.SshKeysDistr= iMonitor 2015-07-06 09:54:02,463 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Disconnect to listener: com.cloud.network.router.Virtu= alNetworkApplianceManagerImpl 2015-07-06 09:54:02,463 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Disconnect to listener: com.cloud.storage.upload.Uploa= dListener 2015-07-06 09:54:02,463 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Disconnect to listener: com.cloud.storage.download.Dow= nloadListener 2015-07-06 09:54:02,463 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Disconnect to listener: com.cloud.consoleproxy.Console= ProxyListener 2015-07-06 09:54:02,464 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Disconnect to listener: com.cloud.network.SshKeysDistr= iMonitor 2015-07-06 09:54:02,464 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Disconnect to listener: com.cloud.network.router.VpcVi= rtualNetworkApplianceManagerImpl 2015-07-06 09:54:02,464 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Sending Disconnect to listener: com.cloud.agent.manager.AgentM= anagerImpl$BehindOnPingListener 2015-07-06 09:54:02,465 DEBUG [c.c.h.Status] (AgentTaskPool-1:ctx-a4685473)= Transition:[Resource state =3D Enabled, Agent event =3D AgentDisconnected,= Host id =3D 1, name =3D SeSolXS01] 2015-07-06 09:54:02,486 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-165= :ctx-fe175294) Seq 5-8999880905346524321: Response Received: 2015-07-06 09:54:02,486 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (DirectAg= ent-165:ctx-fe175294) Seq 5-8999880905346524321: MgmtId 249373720971888: Re= sp: Routing to peer 2015-07-06 09:54:02,505 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-42:= ctx-4b3b996b) Seq 6-1232297448039261404: Executing request 2015-07-06 09:54:02,507 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentTas= kPool-1:ctx-a4685473) Notifying other nodes of to disconnect 2015-07-06 09:54:02,519 DEBUG [c.c.c.ClusterManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Forwarding [{"com.cloud.agent.api.ChangeAgentCommand":{"agentI= d":1,"event":"AgentDisconnected","contextMap":{},"wait":0}}] to 24937372097= 1888 2015-07-06 09:54:02,520 DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Worker-3:= ctx-0a51ef05) Cluster PDU 59778234354585 -> 249373720971888. agent: 1, pdu = seq: 25, pdu ack seq: 0, json: [{"com.cloud.agent.api.ChangeAgentCommand":{= "agentId":1,"event":"AgentDisconnected","contextMap":{},"wait":0}}] 2015-07-06 09:54:02,520 WARN [c.c.r.ResourceManagerImpl] (AgentTaskPool-1:= ctx-a4685473) Unable to connect due to com.cloud.utils.exception.CloudRuntimeException: Unable to connect 1 at com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsO= fConnection(AgentManagerImpl.java:561) at com.cloud.agent.manager.AgentManagerImpl.handleDirectCon= nectAgent(AgentManagerImpl.java:1447) at com.cloud.resource.ResourceManagerImpl.createHostAndAgen= t(ResourceManagerImpl.java:1794) at com.cloud.resource.ResourceManagerImpl.createHostAndAgen= t(ResourceManagerImpl.java:1920) at sun.reflect.GeneratedMethodAccessor263.invoke(Unknown So= urce) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Delegati= ngMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.springframework.aop.support.AopUtils.invokeJoinpoint= UsingReflection(AopUtils.java:317) at org.springframework.aop.framework.ReflectiveMethodInvoca= tion.invokeJoinpoint(ReflectiveMethodInvocation.java:183) at org.springframework.aop.framework.ReflectiveMethodInvoca= tion.proceed(ReflectiveMethodInvocation.java:150) at org.springframework.aop.interceptor.ExposeInvocationInte= rceptor.invoke(ExposeInvocationInterceptor.java:91) at org.springframework.aop.framework.ReflectiveMethodInvoca= tion.proceed(ReflectiveMethodInvocation.java:172) at org.springframework.aop.framework.JdkDynamicAopProxy.inv= oke(JdkDynamicAopProxy.java:204) at com.sun.proxy.$Proxy149.createHostAndAgent(Unknown Sourc= e) at com.cloud.agent.manager.AgentManagerImpl$SimulateStartTa= sk.runInContext(AgentManagerImpl.java:1078) at org.apache.cloudstack.managed.context.ManagedContextRunn= able$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManage= dContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManage= dContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManage= dContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunn= able.run(ManagedContextRunnable.java:46) at java.util.concurrent.ThreadPoolExecutor.runWorker(Thread= PoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Threa= dPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Caused by: java.lang.NullPointerException at com.cloud.capacity.CapacityManagerImpl.updateCapacityFor= Host(CapacityManagerImpl.java:648) at com.cloud.capacity.ComputeCapacityListener.processConnec= t(ComputeCapacityListener.java:66) at com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsO= fConnection(AgentManagerImpl.java:539) ... 22 more 2015-07-06 09:54:02,522 DEBUG [c.c.h.Status] (AgentTaskPool-1:ctx-a4685473)= Transition:[Resource state =3D Enabled, Agent event =3D AgentDisconnected,= Host id =3D 1, name =3D SeSolXS01] 2015-07-06 09:54:02,523 DEBUG [c.c.c.ClusterServiceServletImpl] (Cluster-Wo= rker-3:ctx-0a51ef05) POST http://secure-web.cisco.com/1X7Zx_tVBp5zkXnhtf7v8= RCPEQEVEsCFOQSFwQyocn8nUTksj4DwmCCPBsV8a5mG3gbmJMuMhsKacEuS1du4t4i5GobdHtgT= q67qdDuBAXsJy83xynT3YB99lhxZtmeQH5HRYf0VQtb-kbqjmEcIGacVMcuDCNb-Mc3vGRlmiay= TizV5K0UQpHoNTOvV4OFKB/http%3A%2F%2F172.16.5.190%3A9090%2Fclusterservice re= sponse :true, responding time: 3 ms 2015-07-06 09:54:02,523 DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Worker-3:= ctx-0a51ef05) Cluster PDU 59778234354585 -> 249373720971888 completed. time= : 3ms. agent: 1, pdu seq: 25, pdu ack seq: 0, json: [{"com.cloud.agent.api.= ChangeAgentCommand":{"agentId":1,"event":"AgentDisconnected","contextMap":{= },"wait":0}}] 2015-07-06 09:54:02,561 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-42:= ctx-4b3b996b) Seq 6-1232297448039261404: Response Received: 2015-07-06 09:54:02,561 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (DirectAg= ent-42:ctx-4b3b996b) Seq 6-1232297448039261404: MgmtId 249373720971888: Res= p: Routing to peer 2015-07-06 09:54:02,572 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-1:c= tx-a4685473) Completed creating agent for host 1 2015-07-06 09:54:02,722 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgentCron= Job-236:ctx-79815165) Seq 5-6790865288121286660: Executing request I tried to give maintenance cmd again but failed, 2015-07-06 09:40:01,441 DEBUG [c.c.a.m.AgentManagerImpl] (API-Job-Executor-= 38:ctx-9c1f1f6b job-3786 ctx-c88bc8f8) Can not send command com.cloud.agent= .api.MaintainCommand due to Host 1 is not up 2015-07-06 09:40:01,441 WARN [c.c.r.ResourceManagerImpl] (API-Job-Executor= -38:ctx-9c1f1f6b job-3786 ctx-c88bc8f8) Unable to send MaintainCommand to h= ost: 1 2015-07-06 09:40:01,441 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Execut= or-38:ctx-9c1f1f6b job-3786) Unexpected exception while executing org.apach= e.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd com.cloud.utils.exception.CloudRuntimeException: Unable to prepare for main= tenance host 1 at com.cloud.resource.ResourceManagerImpl.maintain(Resource= ManagerImpl.java:1251) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Meth= od) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethod= AccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Delegati= ngMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.springframework.aop.support.AopUtils.invokeJoinpoint= UsingReflection(AopUtils.java:317) at org.springframework.aop.framework.ReflectiveMethodInvoca= tion.invokeJoinpoint(ReflectiveMethodInvocation.java:183) at org.springframework.aop.framework.ReflectiveMethodInvoca= tion.proceed(ReflectiveMethodInvocation.java:150) at org.springframework.aop.interceptor.ExposeInvocationInte= rceptor.invoke(ExposeInvocationInterceptor.java:91) at org.springframework.aop.framework.ReflectiveMethodInvoca= tion.proceed(ReflectiveMethodInvocation.java:172) at org.springframework.aop.framework.JdkDynamicAopProxy.inv= oke(JdkDynamicAopProxy.java:204) at com.sun.proxy.$Proxy149.maintain(Unknown Source) at org.apache.cloudstack.api.command.admin.host.PrepareForM= aintenanceCmd.execute(PrepareForMaintenanceCmd.java:102) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:= 141) at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDi= spatcher.java:108) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManage= rImpl$5.runInContext(AsyncJobManagerImpl.java:537) at org.apache.cloudstack.managed.context.ManagedContextRunn= able$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManage= dContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManage= dContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManage= dContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunn= able.run(ManagedContextRunnable.java:46) at org.apache.cloudstack.framework.jobs.impl.AsyncJobManage= rImpl$5.run(AsyncJobManagerImpl.java:494) at java.util.concurrent.Executors$RunnableAdapter.call(Exec= utors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(Thread= PoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Threa= dPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) 2015-07-06 09:40:01,443 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Ex= ecutor-38:ctx-9c1f1f6b job-3786) Complete async job-3786, jobStatus: FAILED= , resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResp= onse/null/{"uuidList":[],"errorcode":530,"errortext":"Unable to prepare for= maintenance host 1"} 2015-07-06 09:40:01,445 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Ex= ecutor-38:ctx-9c1f1f6b job-3786) Publish async job-3786 complete on message= bus 2015-07-06 09:40:01,445 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Ex= ecutor-38:ctx-9c1f1f6b job-3786) Wake up jobs related to job-3786 /sonali