cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "angeline shen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CLOUDSTACK-3715) Live Migration of Virtual instances operation is getting timedout on a multinode mgmt setup
Date Fri, 09 Aug 2013 20:58:48 GMT

    [ https://issues.apache.org/jira/browse/CLOUDSTACK-3715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13735296#comment-13735296
] 

angeline shen commented on CLOUDSTACK-3715:
-------------------------------------------

multinode MS setup consist of 4 nodes :
  load balancer
  DB server - separate DB server,  servicing  clustered  MS1  and MS2 
  MS1 - clustered with MS2
  MS2 - clustered with MS1
  hosts  -  XEN   XS 6.2 ,    2 hosts in 1 cluster

1. clustered Management server configuration with MS1  MS2  with load balancer.
2. In advance zone,  1 XEN cluster with 2 XEN hosts.
   create user account  d1user1
3.  login as admin.       create VMs in isolated network
     login as d1user1.   create VMs in isolated network

4. Both MS1 and  MS2  running.  
    login as admin.
    Perform VM live migration for d1user1  and admin VMs.  All completed in few seconds &
successful.
   Perform volume migration for d1user1 and admin volumes.  All completed & successful.

5. MS2 down.    MS1  Up
 login as admin.
    Perform VM live migration for d1user1  and admin VMs.  All completed in few seconds &
successful.
   Perform volume migration for d1user1 and admin volumes.  All completed & successful.

attachment :   MS1 log    MS2 log   DB server   cloud-backup.sql cloud-backup.dmp




                
> Live Migration of Virtual instances operation is getting timedout on a multinode mgmt
setup
> -------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-3715
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3715
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Management Server
>    Affects Versions: 4.2.0
>            Reporter: Sailaja Mada
>            Assignee: Alena Prokharchyk
>            Priority: Blocker
>             Fix For: 4.2.0
>
>         Attachments: apilog.log, management-server.log
>
>
> Setup: Multinode Management setup. 
> Steps:
> 1. Configure Adv Zone with 2 VMWARE clusters each with one hosts with Zone wide primary
storage ( Standard vSwitch cluster) 
> 2. Deploy VM using User account 
> 3. Tried to Live migrate VM from cluster1 (host 1)  to  Cluster 2 (host2 ) 
> Observation:
> 1. Migration took very log time and finally failed saying operation timed out :
> 2013-07-22 17:46:06,288 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-40:job-133
= [ 4a2f6d84-236d-4bdd-a2e5-72fa336e0274 ]) VM state transitted from :Migrating to Running
with event: OperationFailedvm's original host id: 4 new host id: 4 host id before state transition:
1
> 2013-07-22 17:46:06,292 INFO  [vmware.resource.VmwareResource] (DirectAgent-421:10.102.192.18)
VM i-4-9-VM is no longer in vSphere
> 2013-07-22 17:46:06,293 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-421:null)
Seq 1-1311245319: Response Received:
> 2013-07-22 17:46:06,294 DEBUG [agent.transport.Request] (DirectAgent-421:null) Seq 1-1311245319:
Processing:  { Ans: , MgmtId: 94838926819810, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StopAnswer":{"vncPort":0,"result":true,"details":"VM
i-4-9-VM is no longer in vSphere","wait":0}}] }
> 2013-07-22 17:46:06,294 DEBUG [agent.manager.AgentAttache] (DirectAgent-421:null) Seq
1-1311245319: Unable to find listener.
> 2013-07-22 17:46:06,307 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-40:job-133
= [ 4a2f6d84-236d-4bdd-a2e5-72fa336e0274 ]) Hosts's actual total CPU: 9572 and CPU after applying
overprovisioning: 9572
> 2013-07-22 17:46:06,307 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-40:job-133
= [ 4a2f6d84-236d-4bdd-a2e5-72fa336e0274 ]) Hosts's actual total RAM: 17166258176 and RAM
after applying overprovisioning: 17166258176
> 2013-07-22 17:46:06,307 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-40:job-133
= [ 4a2f6d84-236d-4bdd-a2e5-72fa336e0274 ]) release cpu from host: 1, old used: 2000,reserved:
0, actual total: 9572, total with overprovisioning: 9572; new used: 200,reserved:0; movedfromreserved:
false,moveToReserveredfalse
> 2013-07-22 17:46:06,307 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-40:job-133
= [ 4a2f6d84-236d-4bdd-a2e5-72fa336e0274 ]) release mem from host: 1, old used: 2013265920,reserved:
0, total: 17166258176; new used: 2013265920,reserved:0; movedfromreserved: false,moveToReserveredfalse
> 2013-07-22 17:46:06,345 ERROR [cloud.async.AsyncJobManagerImpl] (Job-Executor-40:job-133
= [ 4a2f6d84-236d-4bdd-a2e5-72fa336e0274 ]) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.MigrateVirtualMachineWithVolumeCmd
> com.cloud.utils.exception.CloudRuntimeException: Failed to migrated vm VM[User|newuser1i1]
along with its volumes. com.cloud.exception.AgentUnavailableException: Resource [Host:1] is
unreachable: Host 1: Operation timed out on storage motion for VM[User|newuser1i1]
>         at com.cloud.storage.VolumeManagerImpl.migrateVolumes(VolumeManagerImpl.java:2263)
>         at com.cloud.vm.VirtualMachineManagerImpl.migrateWithStorage(VirtualMachineManagerImpl.java:1780)
>         at com.cloud.vm.UserVmManagerImpl.migrateVirtualMachineWithVolume(UserVmManagerImpl.java:4046)
>         at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>         at org.apache.cloudstack.api.command.admin.vm.MigrateVirtualMachineWithVolumeCmd.execute(MigrateVirtualMachineWithVolumeCmd.java:137)
>         at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
>         at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2013-07-22 17:46:06,350 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-40:job-133
= [ 4a2f6d84-236d-4bdd-a2e5-72fa336e0274 ]) Complete async job-133 = [ 4a2f6d84-236d-4bdd-a2e5-72fa336e0274
], jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to migrated vm
VM[User|newuser1i1] along with its volumes. com.cloud.exception.AgentUnavailableException:
Resource [Host:1] is unreachable: Host 1: Operation timed out on storage motion for VM[User|newuser1i1]

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message