Mailing-List: contact issues-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cloudstack.apache.org
Date: Thu, 31 Oct 2013 23:32:17 +0000 (UTC)
From: "Sheng Yang (JIRA)" <jira@apache.org>
To: cloudstack-issues@incubator.apache.org
Message-ID: <JIRA.12665975.1377714723298.4353.1383262337837@arcas>
In-Reply-To: <JIRA.12665975.1377714723298@arcas>
References: <JIRA.12665975.1377714723298@arcas>
Subject: [jira] [Commented] (CLOUDSTACK-4540) Parallel deployment - Vmware -
 When deploying 30 parallel Vms , 16 Vms fails to get deployed due to
 "VmDataCommand failed due to Exception: java.lang.Exception Message: Timed
 out in waiting SSH execution result"
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CLOUDSTACK-4540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13810861#comment-13810861 ] 

Sheng Yang commented on CLOUDSTACK-4540:
----------------------------------------

After set: vmware.create.full.clone to false, I have no issue when deploying 30 vms at the same time.

If full.clone is true, it would take extremely long time to deploy the multiple vms, which mostly final result in timeout.

BTW, both execute.in.sequence.hypervisor.commands and execute.in.sequence.network.element.commands are set to false in my case.

> Parallel deployment - Vmware - When deploying 30 parallel Vms , 16 Vms fails to get deployed due to "VmDataCommand failed due to Exception: java.lang.Exception Message: Timed out in waiting SSH execution result"
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-4540
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4540
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Management Server
>    Affects Versions: 4.2.0
>         Environment: Build from 4.2-forward.
>            Reporter: Sangeetha Hariharan
>            Assignee: Sheng Yang
>            Priority: Blocker
>             Fix For: 4.2.1
>
>         Attachments: management-server.log
>
>
> Parallel deployment - Vmware - When deploying 30 parallel Vms , 16 Vms fails to get deployed due to "VmDataCommand failed due to Exception: java.lang.Exception
> Message: Timed out in waiting SSH execution result"
> Set up - Advanced zone with 1 Vmware 5.0.0 Esxi host.
> Deploy 30 Vms in parallel.
> 16 out of 30 vms deployed in parallel , failed due to "VmDataCommand failed due to Exception: java.lang.Exception
> Message: Timed out in waiting SSH execution result"
> Following exception seen in Management server logs:
> 2013-08-28 10:26:58,939 ERROR [vmware.resource.VmwareResource] (DirectAgent-21:10.223.58.66) VmDataCommand failed due to Exception: java.lang.Exception
> Message: Timed out in waiting SSH execution result
> java.lang.Exception: Timed out in waiting SSH execution result
>         at com.cloud.utils.ssh.SshHelper.sshExecute(SshHelper.java:166)
>         at com.cloud.utils.ssh.SshHelper.sshExecute(SshHelper.java:37)
>         at com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:2470)
>         at com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:441)
>         at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2013-08-28 10:26:58,940 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-21:null) Seq 1-170983503: Response Received:
> 2013-08-28 10:26:58,941 DEBUG [agent.transport.Request] (DirectAgent-21:null) Seq 1-170983503: Processing:  { Ans: , MgmtId: 7083743249448, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"VmDataCommand failed due to Exception: java.lang.Exception\nMessage: Timed out in waiting SSH execution result\n","wait":0}}] }
> 2013-08-28 10:26:58,941 DEBUG [agent.transport.Request] (Job-Executor-29:job-398 = [ b3a34f25-37b2-4f33-b183-c0ea348d7af9 ]) Seq 1-170983503: Received:  { Ans: , MgmtId: 7083743249448, via: 1, Ver: v1, Flags: 10, { Answer, Answer } }
> 2013-08-28 10:26:58,979 INFO  [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-29:job-398 = [ b3a34f25-37b2-4f33-b183-c0ea348d7af9 ]) Unable to contact resource.
> com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply userdata and password entry on router
>         at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3808)
>         at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyUserData(VirtualNetworkApplianceManagerImpl.java:2993)
>         at com.cloud.network.element.VirtualRouterElement.addPasswordAndUserdata(VirtualRouterElement.java:926)
>         at com.cloud.network.NetworkManagerImpl.prepareElement(NetworkManagerImpl.java:2076)
>         at com.cloud.network.NetworkManagerImpl.prepareNic(NetworkManagerImpl.java:2191)
>         at com.cloud.network.NetworkManagerImpl.prepare(NetworkManagerImpl.java:2127)
>         at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:886)
>         at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:578)
>         at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:227)
>         at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
>         at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3406)
>         at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2966)
>         at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2952)
>         at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>         at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420)
>         at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
>         at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)


--
This message was sent by Atlassian JIRA
(v6.1#6144)