Return-Path: X-Original-To: apmail-cloudstack-issues-archive@www.apache.org Delivered-To: apmail-cloudstack-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 34E1610C80 for ; Mon, 11 Nov 2013 09:27:39 +0000 (UTC) Received: (qmail 565 invoked by uid 500); 11 Nov 2013 09:27:33 -0000 Delivered-To: apmail-cloudstack-issues-archive@cloudstack.apache.org Received: (qmail 522 invoked by uid 500); 11 Nov 2013 09:27:32 -0000 Mailing-List: contact issues-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cloudstack.apache.org Delivered-To: mailing list issues@cloudstack.apache.org Received: (qmail 493 invoked by uid 500); 11 Nov 2013 09:27:28 -0000 Delivered-To: apmail-incubator-cloudstack-issues@incubator.apache.org Received: (qmail 439 invoked by uid 99); 11 Nov 2013 09:27:18 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 11 Nov 2013 09:27:18 +0000 Date: Mon, 11 Nov 2013 09:27:18 +0000 (UTC) From: "Sateesh Chodapuneedi (JIRA)" To: cloudstack-issues@incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Resolved] (CLOUDSTACK-5027) [VMWARE] Failed to Deploy instance when primary storage of a different zone are in maintenance mode MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CLOUDSTACK-5027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sateesh Chodapuneedi resolved CLOUDSTACK-5027. ---------------------------------------------- Resolution: Invalid Following the given steps, again with today's 4.2 branch (commit 951336c3e4510ddfecfe0678be7716e4eade1e98) in the bug description did not help in reproducing the reported problem, which is failure to deploy VM, if another zone's primary storages are sent to maintenance mode. CloudStack maintains isolation between zones. This looks like setup issue rather than a bug. Please reopen if this issue oberved. > [VMWARE] Failed to Deploy instance when primary storage of a different zone are in maintenance mode > --------------------------------------------------------------------------------------------------- > > Key: CLOUDSTACK-5027 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5027 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the default.) > Components: Storage Controller, VMware > Affects Versions: 4.2.1 > Reporter: Sailaja Mada > Assignee: Sateesh Chodapuneedi > Priority: Critical > Fix For: 4.2.1 > > Attachments: failureloigs.rar > > > Steps: > 1. Configure 2 vMWARE zones each with Zone wide primary storages > 2. Moved Zone 1 primary storages into maintenance mode > 3. Tried to deploy VM on second VMWARE zone which has Primary storage up state. > Observation: > [VMWARE] Failed to Deploy instance when primary storage of a different zone are in maintenance mode > 2013-11-03 10:21:07,088 DEBUG [cloud.storage.VolumeManagerImpl] (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) Checking if we need to prepare 1 volumes for VM[DomainRouter|r-30-VM] > 2013-11-03 10:21:07,099 DEBUG [storage.image.TemplateDataFactoryImpl] (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) template 8 is already in store:1, type:Image > 2013-11-03 10:21:07,104 DEBUG [storage.datastore.PrimaryDataStoreImpl] (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) Not found (templateId:8poolId:6) in template_spool_ref, persisting it > 2013-11-03 10:21:07,121 DEBUG [storage.image.TemplateDataFactoryImpl] (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) template 8 is already in store:6, type:Primary > 2013-11-03 10:21:07,123 DEBUG [storage.volume.VolumeServiceImpl] (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) Found template routing-8 in storage pool 6 with VMTemplateStoragePool id: 28 > 2013-11-03 10:21:07,187 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-3:null) SeqA 10-30340: Processing Seq 10-30340: { Cmd , MgmtId: -1, via: 10, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":17,"_loadInfo":"{\n \"connections\": []\n}","wait":0}}] } > 2013-11-03 10:21:07,331 DEBUG [storage.volume.VolumeServiceImpl] (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) Acquire lock on VMTemplateStoragePool 28 with timeout 3600 seconds > 2013-11-03 10:21:07,333 INFO [storage.volume.VolumeServiceImpl] (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) lock is acquired for VMTemplateStoragePool 28 > 2013-11-03 10:21:07,341 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-3:null) SeqA 10-30340: Sending Seq 10-30340: { Ans: , MgmtId: 94838926819810, via: 10, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] } > 2013-11-03 10:21:07,373 DEBUG [storage.motion.AncientDataMotionStrategy] (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) copyAsync inspecting src type TEMPLATE copyAsync inspecting dest type TEMPLATE > 2013-11-03 10:21:07,387 DEBUG [storage.motion.AncientDataMotionStrategy] (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) copy object failed: > java.lang.NullPointerException > at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyObject(AncientDataMotionStrategy.java:210) > at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:411) > at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:58) > at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createBaseImageAsync(VolumeServiceImpl.java:446) > at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:575) > at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2567) > at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2631) > at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:888) > at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:577) > at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2764) > at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1896) > at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1996) > at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1974) > at com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:189) > at com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:2043) > at com.cloud.network.NetworkManagerImpl.implementNetwork(NetworkManagerImpl.java:1948) > at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > at com.cloud.network.NetworkManagerImpl.prepare(NetworkManagerImpl.java:2129) > at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:886) > at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:577) > at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:227) > at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209) > at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3425) > at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2985) > at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2971) > at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420) > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) > at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2013-11-03 10:21:07,479 DEBUG [storage.motion.AncientDataMotionStrategy] (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) copy failed > com.cloud.utils.exception.CloudRuntimeException: java.lang.NullPointerException > at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyObject(AncientDataMotionStrategy.java:231) > at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:411) > at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:58) > at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createBaseImageAsync(VolumeServiceImpl.java:446) > at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:575) > at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2567) > at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2631) > at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:888) > at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:577) > at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2764) > at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1896) > at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1996) > at com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:189) > at com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:2043) > at com.cloud.network.NetworkManagerImpl.implementNetwork(NetworkManagerImpl.java:1948) > at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > at com.cloud.network.NetworkManagerImpl.prepare(NetworkManagerImpl.java:2129) > at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:886) > at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:577) > at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:227) > at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209) > at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3425) > at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2985) > at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2971) > at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420) > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) > at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2013-11-03 10:21:07,615 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) Cleaning up resources for the vm VM[DomainRouter|r-30-VM] in Starting state > 2013-11-03 10:21:07,616 DEBUG [agent.transport.Request] (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) Seq 9-894185288: Sending { Cmd , MgmtId: 94838926819810, via: 9, Ver: v1, Flags: 100111, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":true,"vmName":"r-30-VM","wait":0}}] } > 2013-11-03 10:21:07,617 DEBUG [agent.transport.Request] (Job-Executor-17:job-155 = [ d4cadcb1-7e06-4ee2-9d08-0585964fc934 ]) Seq 9-894185288: Executing: { Cmd , MgmtId: 94838926819810, via: 9, Ver: v1, Flags: 100111, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":true,"vmName":"r-30-VM","wait":0}}] } > 2013-11-03 10:21:07,646 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-148:null) Seq 9-894185288: Executing request > 2013-11-03 10:21:07,647 INFO [vmware.resource.VmwareResource] (DirectAgent-148:10.102.192.18) Executing resource StopCommand: {"isProxy":false,"executeInSequence":true,"vmName":"r-30-VM","wait":0} > 2013-11-03 10:21:07,676 DEBUG [vmware.mo.HostMO] (DirectAgent-148:10.102.192.18) find VM r-30-VM on host -- This message was sent by Atlassian JIRA (v6.1#6144)