cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kelven Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CLOUDSTACK-3568) [Automation]Parallel deployment - Vmware - When deploying 30 parallel Vms , we see ~15 Vms fail in org.apache.cloudstack.storage.command.CopyCommand.
Date Sat, 10 Aug 2013 00:49:48 GMT

    [ https://issues.apache.org/jira/browse/CLOUDSTACK-3568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13735615#comment-13735615
] 

Kelven Yang commented on CLOUDSTACK-3568:
-----------------------------------------

We need to manage a pool of VmwareContext objects and manage API calls to vCenter through
acquired VmwareContext object to eliminate sharing of vCenter session among multiple threads.


I'll work on this change and hopefully to make it available as soon as possible
                
> [Automation]Parallel deployment - Vmware - When deploying 30 parallel Vms , we see ~15
Vms fail in org.apache.cloudstack.storage.command.CopyCommand.
> -----------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-3568
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3568
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Management Server, VMware
>    Affects Versions: 4.2.0
>         Environment: Build from 4.2
>            Reporter: Sangeetha Hariharan
>            Assignee: Kelven Yang
>            Priority: Blocker
>             Fix For: 4.2.0
>
>         Attachments: management-server.log
>
>
> Parallel deployment - Vmware - When deploying 30 parallel Vms , we see ~15 Vms fail in
org.apache.cloudstack.storage.command.CopyCommand.
> Steps to reproduce the problem:
> Set up - Advanced zone with Vmware ESXI, 5.0.0 host.
> Deploy 30 Vms in parallel in a network that is already implemented (router is already
running).
> Only 9 vms were successfully deployed.
> ~15 Vms failed in org.apache.cloudstack.storage.command.CopyCommand because of "ROOT-106-104/ROOT-106-104.vmdk
was not found" error.
>  mysql> select count(*),state,type from vm_instance where name like "hello%" group
by state,type ;
> +----------+---------+------+
> | count(*) | state   | type |
> +----------+---------+------+
> |       42 | Error   | User |
> |        9 | Running | User |
> +----------+---------+------+
> 2 rows in set (0.00 sec)
> Following exception seen in management server logs:
> 2013-07-16 15:47:55,099 ERROR [storage.resource.VmwareStorageProcessor] (DirectAgent-71:10.223.57.66)
CreateCommand failed due to Exception: java.lang.RuntimeException
> Message: File [7caed722555d3f71879476c5e8d8d5ad] ROOT-106-104/ROOT-106-104.vmdk was not
found
> java.lang.RuntimeException: File [7caed722555d3f71879476c5e8d8d5ad] ROOT-106-104/ROOT-106-104.vmdk
was not found
>         at com.cloud.hypervisor.vmware.util.VmwareClient.waitForTask(VmwareClient.java:290)
>         at com.cloud.hypervisor.vmware.mo.HostDatastoreBrowserMO.searchDatastore(HostDatastoreBrowserMO.java:57)
>         at com.cloud.hypervisor.vmware.mo.HostDatastoreBrowserMO.searchDatastore(HostDatastoreBrowserMO.java:78)
>         at com.cloud.hypervisor.vmware.mo.DatastoreMO.folderExists(DatastoreMO.java:319)
>         at com.cloud.storage.resource.VmwareStorageProcessor.createVMLinkedClone(VmwareStorageProcessor.java:279)
>         at com.cloud.storage.resource.VmwareStorageProcessor.cloneVolumeFromBaseTemplate(VmwareStorageProcessor.java:393)
>         at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:73)
>         at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:49)
>         at com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:565)
>         at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2013-07-16 15:47:55,100 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-71:null)
Seq 1-227737729: Response Received:
> 2013-07-16 15:47:55,100 DEBUG [agent.transport.Request] (DirectAgent-71:null) Seq 1-227737729:
Processing:  { Ans: , MgmtId: 7647994577963, via: 1, Ver: v1, Flags: 10, [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"java.lang.RuntimeException:
File [7caed722555d3f71879476c5e8d8d5ad] ROOT-106-104/ROOT-106-104.vmdk was not found","wait":0}}]
}
> 2013-07-16 15:47:55,100 DEBUG [agent.transport.Request] (Job-Executor-8:job-101 = [ cad3c4ea-3d40-46f4-bce5-26d60d4af7c1
]) Seq 1-227737729: Received:  { Ans: , MgmtId: 7647994577963, via: 1, Ver: v1, Flags: 10,
{ CopyCmdAnswer } }
> mysql> select count(*),state,type from vm_instance where name like "hello%" group
by state,type ;
> +----------+---------+------+
> | count(*) | state   | type |
> +----------+---------+------+
> |       42 | Error   | User |
> |        9 | Running | User |
> +----------+---------+------+
> 2 rows in set (0.00 sec)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message