cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Denis Finko (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CLOUDSTACK-7757) CloudStack don't remove VM from VMware
Date Tue, 21 Oct 2014 09:46:33 GMT
Denis Finko created CLOUDSTACK-7757:
---------------------------------------

             Summary: CloudStack don't remove VM from VMware
                 Key: CLOUDSTACK-7757
                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-7757
             Project: CloudStack
          Issue Type: Bug
      Security Level: Public (Anyone can view this level - this is the default.)
    Affects Versions: 4.2.1
         Environment: VMware ESXi 5.1.0
            Reporter: Denis Finko


Hello CloudStack community,

I have found a lot of old VMs that were removed from CloudStack but still present in VMware.
After that I created 6 similar VMs in CloudStack and tried to remove then from user interface
with 'Expunge' mark:
http://take.ms/vZ4AE

I was surprised that only 3 VMs from 6 were removed from VMware:

-------------------+-----------------------+------------------------------------------------+
 name             |   removed time      | Is VM was removed from VMware
-------------------+-----------------------+------------------------------------------------+
test-2010-3 |       07:46:53        |   NO
-------------------+-----------------------+------------------------------------------------+
test-2010-4 |       07:49:45        |   YES
-------------------+-----------------------+------------------------------------------------+
test-2010-5 |       07:55:32        |   YES
-------------------+-----------------------+------------------------------------------------+
test-2110-1 |       07:57:08        |   NO
-------------------+-----------------------+------------------------------------------------+
test-2110-2 |       08:00:44        |   NO
-------------------+-----------------------+------------------------------------------------+
test-2110-3 |       08:03:06        |   YES
-------------------+-----------------------+------------------------------------------------+

I have checked management-server.log for 'test-2110-2 ' and 'test-2110-3' VMs and their looks
similar:

logs for test-2110-2 (that VM wasn't removed from VMware environment):
2014-10-21 04:00:44,190 DEBUG [cloud.async.AsyncJobManagerImpl] (ajp-20400-5:null) submit
async job-7170 = [ b37ded66-33f7-4782-a36a-b7559d7d1462 ], details: AsyncJobVO {id:7170, userId:
2, accountId: 2, sessionKey: null, instanceType: VirtualMachine, instanceId: 2185, cmd: org.apache.cloudstack.api.command.user.vm.DestroyVMCmd,
cmdOriginator: null, cmdInfo: {"id":"b84498ba-bd20-4390-b239-0123fa566cac","response":"json","sessionkey":"vjRevp3+g1aH20gH0JkG8EVRSfg\u005d","cmdEventType":"VM.DESTROY","ctxUserId":"2","httpmethod":"GET","_":"1413878444174","ctxAccountId":"2","expunge":"true","ctxStartEventId":"407103"},
cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0, processStatus: 0, resultCode:
0, result: null, initMsid: 345048632606, completeMsid: null, lastUpdated: null, lastPolled:
null, created: null}
2014-10-21 04:00:44,190 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-28:job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) Executing org.apache.cloudstack.api.command.user.vm.DestroyVMCmd
for job-7170 = [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]
2014-10-21 04:00:44,213 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-28:job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) Destroying vm VM[User|test-2110-2]
2014-10-21 04:00:44,213 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-28:job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) VM is already stopped: VM[User|test-2110-2]
2014-10-21 04:00:44,223 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-28:job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) VM state transitted from :Stopped to Destroyed
with event: DestroyRequestedvm's original host id: 22 new host id: null host id before state
transition: null
2014-10-21 04:00:44,231 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-28:job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) Hosts's actual total CPU: 67168 and CPU after
applying overprovisioning: 67168
2014-10-21 04:00:44,231 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-28:job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) Hosts's actual total RAM: 137391218688 and RAM
after applying overprovisioning: 137391218688
2014-10-21 04:00:44,231 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-28:job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) release cpu from host: 22, old used: 56486,reserved:
1000, actual total: 67168, total with overprovisioning: 67168; new used: 56486,reserved:0;
movedfromreserved: true,moveToReserveredfalse
2014-10-21 04:00:44,231 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-28:job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) release mem from host: 22, old used: 44023414784,reserved:
2147483648, total: 137391218688; new used: 44023414784,reserved:0; movedfromreserved: true,moveToReserveredfalse
2014-10-21 04:00:44,256 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-28:job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) Stopped called on VM[User|test-2110-2] but the
state is Destroyed
2014-10-21 04:00:44,261 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-28:job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) VM state transitted from :Destroyed to Expunging
with event: ExpungeOperationvm's original host id: 22 new host id: null host id before state
transition: null
2014-10-21 04:00:44,261 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-28:job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) Destroying vm VM[User|test-2110-2]
2014-10-21 04:00:44,261 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-28:job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) Cleaning up NICS
2014-10-21 04:00:44,264 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-28:job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) Cleaning network for vm: 2185
2014-10-21 04:00:44,270 DEBUG [cloud.network.NetworkModelImpl] (Job-Executor-28:job-7170 =
[ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) Service SecurityGroup is not supported in the network
id=207
2014-10-21 04:00:44,277 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-28:job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) Asking VirtualRouter to release Nic[2615-2185-null-xx.xx.xx.xx]
2014-10-21 04:00:44,307 DEBUG [network.guru.DirectNetworkGuru] (Job-Executor-28:job-7170 =
[ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) Deallocate network: networkId: 207, ip: xx.xx.xx.xx
2014-10-21 04:00:44,317 DEBUG [network.guru.DirectNetworkGuru] (Job-Executor-28:job-7170 =
[ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) remove nic 2615 secondary ip 
2014-10-21 04:00:44,319 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-28:job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) Removed nic id=2615
2014-10-21 04:00:44,320 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-28:job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) Revoving nic secondary ip entry ...
2014-10-21 04:00:44,323 DEBUG [cloud.storage.VolumeManagerImpl] (Job-Executor-28:job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) Cleaning storage for vm: 2185
2014-10-21 04:00:44,390 DEBUG [agent.manager.ClusteredAgentAttache] (Job-Executor-28:job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) Seq 14-1013397135: Forwarding Seq 14-1013397135:
 { Cmd , MgmtId: 345048632606, via: 14, Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"d30ff69c-e473-470c-acbc-5aef797ba2ff","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"096b3b6e-481d-33b4-9b34-7178612a2535","id":2,"poolType":"VMFS","host":"VMFS
datastore: /DC/SATA1","path":"/DC/SATA1","port":0}},"name":"ROOT-2185","size":10737418240,"path":"ROOT-2185","volumeId":2236,"vmName":"i-2-2185-VM","accountId":2,"chainInfo":"{\"diskDeviceBusName\":\"scsi0:0\",\"diskChain\":[\"[SATA1]
i-2-2185-VM/ROOT-2185.vmdk\"]}","format":"OVA","id":2236,"hypervisorType":"VMware"}},"wait":0}}]
} to 345048630655
2014-10-21 04:00:44,564 DEBUG [agent.transport.Request] (Job-Executor-28:job-7170 = [ b37ded66-33f7-4782-a36a-b7559d7d1462
]) Seq 14-1013397135: Received:  { Ans: , MgmtId: 345048632606, via: 14, Ver: v1, Flags: 10,
{ Answer } }
2014-10-21 04:00:44,571 INFO  [storage.volume.VolumeServiceImpl] (Job-Executor-28:job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) Volume 2236 is not referred anywhere, remove it
from volumes table
2014-10-21 04:00:44,585 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-28:job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) Expunged VM[User|test-2110-2]
2014-10-21 04:00:44,601 DEBUG [cloud.vm.UserVmManagerImpl] (Job-Executor-28:job-7170 = [ b37ded66-33f7-4782-a36a-b7559d7d1462
]) Starting cleaning up vm VM[User|test-2110-2] resources...
2014-10-21 04:00:44,608 DEBUG [network.firewall.FirewallManagerImpl] (Job-Executor-28:job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) No firewall rules are found for vm id=2185
2014-10-21 04:00:44,608 DEBUG [cloud.vm.UserVmManagerImpl] (Job-Executor-28:job-7170 = [ b37ded66-33f7-4782-a36a-b7559d7d1462
]) Firewall rules are removed successfully as a part of vm id=2185 expunge
2014-10-21 04:00:44,610 DEBUG [network.rules.RulesManagerImpl] (Job-Executor-28:job-7170 =
[ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) No port forwarding rules are found for vm id=2185
2014-10-21 04:00:44,611 DEBUG [cloud.vm.UserVmManagerImpl] (Job-Executor-28:job-7170 = [ b37ded66-33f7-4782-a36a-b7559d7d1462
]) Port forwarding rules are removed successfully as a part of vm id=2185 expunge
2014-10-21 04:00:44,612 DEBUG [cloud.vm.UserVmManagerImpl] (Job-Executor-28:job-7170 = [ b37ded66-33f7-4782-a36a-b7559d7d1462
]) Removed vm id=2185 from all load balancers as a part of expunge process
2014-10-21 04:00:44,613 DEBUG [cloud.vm.UserVmManagerImpl] (Job-Executor-28:job-7170 = [ b37ded66-33f7-4782-a36a-b7559d7d1462
]) Successfully cleaned up vm VM[User|test-2110-2] resources as a part of expunge process
2014-10-21 04:00:44,625 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-28:job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) Complete async job-7170 = [ b37ded66-33f7-4782-a36a-b7559d7d1462
], jobStatus: 1, resultCode: 0, result: org.apache.cloudstack.api.response.UserVmResponse@58f21bb9
2014-10-21 04:00:44,627 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-28:job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]) Done executing org.apache.cloudstack.api.command.user.vm.DestroyVMCmd
for job-7170 = [ b37ded66-33f7-4782-a36a-b7559d7d1462 ]
2014-10-21 04:00:47,388 DEBUG [cloud.async.AsyncJobManagerImpl] (ajp-20400-5:null) Async job-7170
= [ b37ded66-33f7-4782-a36a-b7559d7d1462 ] completed

Please don't tell me that I am need to change expunge.delay or expunge.interval in Global
Settings. It's not helpful in this issue. I have already tried.

How I can remove old VMs from VMware environment via CloudStack?

Thank you!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message