Return-Path: X-Original-To: apmail-cloudstack-issues-archive@www.apache.org Delivered-To: apmail-cloudstack-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8C80D10BDF for ; Wed, 31 Jul 2013 18:51:49 +0000 (UTC) Received: (qmail 94699 invoked by uid 500); 31 Jul 2013 18:51:49 -0000 Delivered-To: apmail-cloudstack-issues-archive@cloudstack.apache.org Received: (qmail 94543 invoked by uid 500); 31 Jul 2013 18:51:49 -0000 Mailing-List: contact issues-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cloudstack.apache.org Delivered-To: mailing list issues@cloudstack.apache.org Received: (qmail 94526 invoked by uid 500); 31 Jul 2013 18:51:48 -0000 Delivered-To: apmail-incubator-cloudstack-issues@incubator.apache.org Received: (qmail 94521 invoked by uid 99); 31 Jul 2013 18:51:48 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 31 Jul 2013 18:51:48 +0000 Date: Wed, 31 Jul 2013 18:51:48 +0000 (UTC) From: "Min Chen (JIRA)" To: cloudstack-issues@incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Assigned] (CLOUDSTACK-3977) vm deployement is failing ;: Resource [Host:5] is unreachable: Host 5: Unable to send class org.apache.cloudstack.storage.command.CopyCommand because agent is in maintenance mode MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CLOUDSTACK-3977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Min Chen reassigned CLOUDSTACK-3977: ------------------------------------ Assignee: Min Chen > vm deployement is failing ;: Resource [Host:5] is unreachable: Host 5: Unable to send class org.apache.cloudstack.storage.command.CopyCommand because agent is in maintenance mode > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > Key: CLOUDSTACK-3977 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3977 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the default.) > Components: Storage Controller > Affects Versions: 4.2.0 > Environment: hypervisor:xen6.1 > Reporter: prashant kumar mishra > Assignee: Min Chen > Priority: Critical > Fix For: 4.2.0 > > Attachments: Logs_DB.rar > > > Steps to reproduce > ------------------------------ > 1-preapre a CS setup with xen6.1 > 2-deploy a vm > 3-add another host say h2 to same cluster > 4-put h2 in maintenance mode > 5-try to deploy one more vm > Expected > -------------- > deploy vm should be successful > Actual > --------- > deploy vm is failing > My observation > ---------------------- > ->was able to deploy vm when h2 was not in maintenance mode > -->was able deploy vm when i removed h2 from cluster > Logs > -------- > 2013-07-31 13:35:08,675 DEBUG [storage.image.TemplateDataFactoryImpl] (Job-Executor-77:job-291 = [ 6af942b2-9301-4aef-afff-3278581cc917 ]) template 5 is already in store:1, type:Image > 2013-07-31 13:35:08,681 DEBUG [storage.image.TemplateDataFactoryImpl] (Job-Executor-77:job-291 = [ 6af942b2-9301-4aef-afff-3278581cc917 ]) template 5 is already in store:2, type:Primary > 2013-07-31 13:35:08,710 DEBUG [storage.motion.AncientDataMotionStrategy] (Job-Executor-77:job-291 = [ 6af942b2-9301-4aef-afff-3278581cc917 ]) copyAsync inspecting src type TEMPLATE copyAsync inspecting dest type VOLUME > 2013-07-31 13:35:08,724 DEBUG [cloudstack.storage.RemoteHostEndPoint] (Job-Executor-77:job-291 = [ 6af942b2-9301-4aef-afff-3278581cc917 ]) Failed to send command, due to Agent:5, com.cloud.exception.AgentUnavailableException: Resource [Host:5] is unreachable: Host 5: Unable to send class org.apache.cloudstack.storage.command.CopyCommand because agent is in maintenance mode > 2013-07-31 13:35:08,725 DEBUG [storage.motion.AncientDataMotionStrategy] (Job-Executor-77:job-291 = [ 6af942b2-9301-4aef-afff-3278581cc917 ]) Failed to send to storage pool > com.cloud.utils.exception.CloudRuntimeException: Failed to send command, due to Agent:5, com.cloud.exception.AgentUnavailableException: Resource [Host:5] is unreachable: Host 5: Unable to send class org.apache.cloudstack.storage.command.CopyCommand because agent is in maintenance mode > at org.apache.cloudstack.storage.RemoteHostEndPoint.sendMessage(RemoteHostEndPoint.java:104) > at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.cloneVolume(AncientDataMotionStrategy.java:270) > at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:373) > at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55) > at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromBaseImageAsync(VolumeServiceImpl.java:501) > at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:538) > at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2497) > at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2554) > at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:913) > at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:618) > at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:227) > at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209) > at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3329) > at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2889) > at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2875) > at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420) > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) > at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2013-07-31 13:35:08,726 DEBUG [storage.motion.AncientDataMotionStrategy] (Job-Executor-77:job-291 = [ 6af942b2-9301-4aef-afff-3278581cc917 ]) copy failed > com.cloud.utils.exception.CloudRuntimeException: Failed to send to storage pool > at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.cloneVolume(AncientDataMotionStrategy.java:274) > at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:373) > at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55) > at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromBaseImageAsync(VolumeServiceImpl.java:501) > at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:538) > at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2497) > at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2554) > at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:913) > at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:618) > at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:227) > at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209) > at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3329) > at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2889) > at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2875) > at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420) > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) > at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > Caused by: com.cloud.utils.exception.CloudRuntimeException: Failed to send command, due to Agent:5, com.cloud.exception.AgentUnavailableException: Resource [Host:5] is unreachable: Host 5: Unable to send class org.apache.cloudstack.storage.command.CopyCommand because agent is in maintenance mode > at org.apache.cloudstack.storage.RemoteHostEndPoint.sendMessage(RemoteHostEndPoint.java:104) > at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.cloneVolume(AncientDataMotionStrategy.java:270) > ... 27 more > 2013-07-31 13:35:08,740 WARN [storage.datastore.ObjectInDataStoreManagerImpl] (Job-Executor-77:job-291 = [ 6af942b2-9301-4aef-afff-3278581cc917 ]) Unsupported data object (VOLUME, org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@181c6ea7) > 2013-07-31 13:35:08,741 DEBUG [cloud.storage.VolumeManagerImpl] (Job-Executor-77:job-291 = [ 6af942b2-9301-4aef-afff-3278581cc917 ]) Unable to create Vol[69|vm=72|ROOT]:com.cloud.utils.exception.CloudRuntimeException: Failed to send to storage pool > 2013-07-31 13:35:08,741 INFO [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-77:job-291 = [ 6af942b2-9301-4aef-afff-3278581cc917 ]) Unable to contact resource. > com.cloud.exception.StorageUnavailableException: Resource [StoragePool:2] is unreachable: Unable to create Vol[69|vm=72|ROOT]:com.cloud.utils.exception.CloudRuntimeException: Failed to send to storage pool > at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2505) > at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2554) > at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:913) > at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:618) > at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:227) > at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209) > at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3329) > at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2889) > at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2875) > at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420) > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) > at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2013-07-31 13:35:08,751 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-77:job-291 = [ 6af942b2-9301-4aef-afff-3278581cc917 ]) Cleaning up resources for the vm VM[User|test342] in Starting state > 2013-07-31 13:35:08,754 DEBUG [agent.transport.Request] (Job-Executor-77:job-291 = [ 6af942b2-9301-4aef-afff-3278581cc917 ]) Seq 1-1484588932: Sending { Cmd , MgmtId: 7635042566263, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"i-2-72-VM","wait":0}}] } -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira