cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjay Tripathi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CLOUDSTACK-7799) [Accounts] Account deletion failed to remove all the entities related to it in case of failure in deleting one entity
Date Thu, 06 Nov 2014 09:50:34 GMT

    [ https://issues.apache.org/jira/browse/CLOUDSTACK-7799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200036#comment-14200036
] 

Sanjay Tripathi commented on CLOUDSTACK-7799:
---------------------------------------------

Hi Sanjeev,

If one entity fails to delete then CS set the accountCleanUpNeeded to true and deletes the
account entry from DB. Later account cleanup thread picks up this account for cleaning and
deletes all the entities associated with this account.

Admin can set the "account.cleanup.interval" gc parameter to cleanup the accounts frequently.

> [Accounts] Account deletion failed to remove all the entities related to it in case of
failure in deleting one entity
> ---------------------------------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-7799
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-7799
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Management Server
>    Affects Versions: 4.5.0
>         Environment: Latest build from 4.5 with commit [root@BPKxDmS ~]# cloudstack-sccs
> 385c4f673dfbd1fd326e539625e2c06db4cdc27d
>            Reporter: Sanjeev N
>            Assignee: Sanjay Tripathi
>            Priority: Critical
>             Fix For: 4.5.0
>
>         Attachments: cloud.dmp, management-server.rar
>
>
> [Critical][Accounts] Account deletion failed to remove all the entities related to it
in case of failure in deleting one entity
> Steps to Reproduce:
> ================
> 1.Bring up CS in advanced zone with one vmware cluster
> 2.Create a user account 
> 3.With the user account deploy few (4-5) vms 
> 4.Take snapshots on all of the vms root disks
> 5.Deploy another vm and simulate snapshot failure operation(In my case I tried snapshot
operation with quiesce option set to true) so that snapshot will be in "Allocated" state
> 6.Delete this user account
> Expected Behavior:
> ==============
> Whatever may the state of the snapshot account deletion shall clean all the objects related
that account
> Actual Behavior:
> =============
> Account deletion started deleting snapshots. Since one of the snapshots is in Allocated
state it failed to delete that snapshot and the Account clean up job ended there. So vms and
networks related to the accounts are still there but the account was deleted.
> Here is the log snippet:
> =================
> 2014-10-28 15:16:38,822 DEBUG [c.c.a.ApiServlet] (catalina-exec-5:ctx-1c16bc72) ===START===
 10.252.193.8 -- GET  command=deleteAccount&response=json&sessionkey=EpI1EuP5ZFF0VKispQyNb9AxWF4%3D&id=83f26103-9d83-4e4e-8bcc-c282c2acb498&_=1414470667367
> 2014-10-28 15:16:38,974 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (catalina-exec-5:ctx-1c16bc72
ctx-51140d28) submit async job-53, details: AsyncJobVO {id:53, userId: 2, accountId: 2, instanceType:
Account, instanceId: null, cmd: org.apache.cloudstack.api.command.admin.account.DeleteAccountCmd,
cmdInfo: {"response":"json","id":"83f26103-9d83-4e4e-8bcc-c282c2acb498","sessionkey":"EpI1EuP5ZFF0VKispQyNb9AxWF4\u003d","ctxDetails":"{\"com.cloud.user.Account\":\"83f26103-9d83-4e4e-8bcc-c282c2acb498\"}","cmdEventType":"ACCOUNT.DELETE","ctxUserId":"2","httpmethod":"GET","_":"1414470667367","uuid":"83f26103-9d83-4e4e-8bcc-c282c2acb498","ctxAccountId":"2","ctxStartEventId":"118"},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid:
6637838401571, completeMsid: null, lastUpdated: null, lastPolled: null, created: null}
> 2014-10-28 15:16:38,978 DEBUG [c.c.a.ApiServlet] (catalina-exec-5:ctx-1c16bc72 ctx-51140d28)
===END===  10.252.193.8 -- GET  command=deleteAccount&response=json&sessionkey=EpI1EuP5ZFF0VKispQyNb9AxWF4%3D&id=83f26103-9d83-4e4e-8bcc-c282c2acb498&_=1414470667367
> 2014-10-28 15:16:38,996 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (API-Job-Executor-30:ctx-0b9151e2
job-53) Add job-53 into job monitoring
> 2014-10-28 15:16:38,996 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-30:ctx-0b9151e2
job-53) Executing AsyncJobVO {id:53, userId: 2, accountId: 2, instanceType: Account, instanceId:
null, cmd: org.apache.cloudstack.api.command.admin.account.DeleteAccountCmd, cmdInfo: {"response":"json","id":"83f26103-9d83-4e4e-8bcc-c282c2acb498","sessionkey":"EpI1EuP5ZFF0VKispQyNb9AxWF4\u003d","ctxDetails":"{\"com.cloud.user.Account\":\"83f26103-9d83-4e4e-8bcc-c282c2acb498\"}","cmdEventType":"ACCOUNT.DELETE","ctxUserId":"2","httpmethod":"GET","_":"1414470667367","uuid":"83f26103-9d83-4e4e-8bcc-c282c2acb498","ctxAccountId":"2","ctxStartEventId":"118"},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid:
6637838401571, completeMsid: null, lastUpdated: null, lastPolled: null, created: null}
> 2014-10-28 15:16:39,153 DEBUG [c.c.u.AccountManagerImpl] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) Removed account 4
> 2014-10-28 15:16:39,287 DEBUG [c.c.a.t.Request] (API-Job-Executor-30:ctx-0b9151e2 job-53
ctx-3bc74f24) Seq 4-4574812796478292948: Sending  { Cmd , MgmtId: 6637838401571, via: 4(s-2-QA),
Ver: v1, Flags: 100111, [{"com.cloud.agent.api.DeleteSnapshotsDirCommand":{"store":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/sanjeev/sec_45","_role":"Image"}},"directory":"snapshots/4/5","wait":0}}]
}
> 2014-10-28 15:16:40,556 DEBUG [c.c.a.t.Request] (AgentManager-Handler-12:null) Seq 4-4574812796478292948:
Processing:  { Ans: , MgmtId: 6637838401571, via: 4, Ver: v1, Flags: 110, [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}]
}
> 2014-10-28 15:16:40,556 DEBUG [c.c.a.t.Request] (API-Job-Executor-30:ctx-0b9151e2 job-53
ctx-3bc74f24) Seq 4-4574812796478292948: Received:  { Ans: , MgmtId: 6637838401571, via: 4,
Ver: v1, Flags: 110, { Answer } }
> 2014-10-28 15:16:40,557 DEBUG [c.c.s.s.SnapshotManagerImpl] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) Deleted all snapshots for volume: 5 under account: 4
> 2014-10-28 15:16:40,561 DEBUG [c.c.a.m.AgentAttache] (AgentManager-Handler-12:null) Seq
4-4574812796478292948: No more commands found
> 2014-10-28 15:16:40,628 DEBUG [o.a.c.s.s.XenserverSnapshotStrategy] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) delete snapshot chain for snapshot: 1
> 2014-10-28 15:16:40,637 DEBUG [o.a.c.s.s.XenserverSnapshotStrategy] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) Snapshot: 1 doesn't have children, so it's ok to delete it and its parents
> 2014-10-28 15:16:40,713 DEBUG [c.c.a.t.Request] (API-Job-Executor-30:ctx-0b9151e2 job-53
ctx-3bc74f24) Seq 4-4574812796478292949: Sending  { Cmd , MgmtId: 6637838401571, via: 4(s-2-QA),
Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/4/5/a2b6fc7f-c086-4494-aa89-d271f0b96d36/a2b6fc7f-c086-4494-aa89-d271f0b96d36","volume":{"uuid":"280c4f37-87ff-4b62-b1f5-5c610e90531d","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"adc69295-88b2-3db8-9e55-fe7db3847af9","id":1,"poolType":"NetworkFilesystem","host":"10.147.28.7","path":"/export/home/sanjeev/pri_cluster","port":2049,"url":"NetworkFilesystem://10.147.28.7/export/home/sanjeev/pri_cluster/?ROLE=Primary&STOREUUID=adc69295-88b2-3db8-9e55-fe7db3847af9"}},"name":"ROOT-5","size":2147483648,"path":"ROOT-5","volumeId":5,"vmName":"i-4-5-QA","accountId":4,"chainInfo":"{\"diskDeviceBusName\":\"ide0:1\",\"diskChain\":[\"[adc6929588b23db89e55fe7db3847af9]
i-4-5-QA/ROOT-5.vmdk\",\"[adc6929588b23db89e55fe7db3847af9] 0bc482b12d62345e9fca3cbce554086c/0bc482b12d62345e9fca3cbce554086c.vmdk\"]}","format":"OVA","provisioningType":"THIN","id":5,"deviceId":0,"hypervisorType":"VMware"},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/sanjeev/sec_45","_role":"Image"}},"vmName":"i-4-5-QA","name":"v1_ROOT-5_20141027213127","hypervisorType":"VMware","id":1,"quiescevm":false,"physicalSize":0}},"wait":0}}]
}
> 2014-10-28 15:16:40,768 DEBUG [c.c.a.t.Request] (AgentManager-Handler-17:null) Seq 4-4574812796478292949:
Processing:  { Ans: , MgmtId: 6637838401571, via: 4, Ver: v1, Flags: 10, [{"com.cloud.agent.api.Answer":{"result":true,"details":"snapshot
directory a2b6fc7f-c086-4494-aa89-d271f0b96d36 doesn't exist","wait":0}}] }
> 2014-10-28 15:16:40,768 DEBUG [c.c.a.t.Request] (API-Job-Executor-30:ctx-0b9151e2 job-53
ctx-3bc74f24) Seq 4-4574812796478292949: Received:  { Ans: , MgmtId: 6637838401571, via: 4,
Ver: v1, Flags: 10, { Answer } }
> 2014-10-28 15:16:40,874 DEBUG [c.c.a.t.Request] (API-Job-Executor-30:ctx-0b9151e2 job-53
ctx-3bc74f24) Seq 4-4574812796478292950: Sending  { Cmd , MgmtId: 6637838401571, via: 4(s-2-QA),
Ver: v1, Flags: 100111, [{"com.cloud.agent.api.DeleteSnapshotsDirCommand":{"store":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/sanjeev/sec_45","_role":"Image"}},"directory":"snapshots/4/7","wait":0}}]
}
> 2014-10-28 15:16:42,048 DEBUG [c.c.a.ApiServlet] (catalina-exec-6:ctx-452a9eca) ===START===
 10.252.193.8 -- GET  command=queryAsyncJobResult&jobId=2c5b0fa0-ef16-4a61-bd64-7cae6acfdaa4&response=json&sessionkey=EpI1EuP5ZFF0VKispQyNb9AxWF4%3D&_=1414470670595
> 2014-10-28 15:16:42,100 DEBUG [c.c.a.ApiServlet] (catalina-exec-6:ctx-452a9eca ctx-9e36e958)
===END===  10.252.193.8 -- GET  command=queryAsyncJobResult&jobId=2c5b0fa0-ef16-4a61-bd64-7cae6acfdaa4&response=json&sessionkey=EpI1EuP5ZFF0VKispQyNb9AxWF4%3D&_=1414470670595
> 2014-10-28 15:16:42,267 DEBUG [c.c.a.t.Request] (AgentManager-Handler-15:null) Seq 4-4574812796478292950:
Processing:  { Ans: , MgmtId: 6637838401571, via: 4, Ver: v1, Flags: 110, [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}]
}
> 2014-10-28 15:16:42,268 DEBUG [c.c.a.t.Request] (API-Job-Executor-30:ctx-0b9151e2 job-53
ctx-3bc74f24) Seq 4-4574812796478292950: Received:  { Ans: , MgmtId: 6637838401571, via: 4,
Ver: v1, Flags: 110, { Answer } }
> 2014-10-28 15:16:42,268 DEBUG [c.c.s.s.SnapshotManagerImpl] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) Deleted all snapshots for volume: 7 under account: 4
> 2014-10-28 15:16:42,271 DEBUG [c.c.a.m.AgentAttache] (AgentManager-Handler-15:null) Seq
4-4574812796478292950: No more commands found
> 2014-10-28 15:16:42,311 DEBUG [o.a.c.s.s.XenserverSnapshotStrategy] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) delete snapshot chain for snapshot: 2
> 2014-10-28 15:16:42,314 DEBUG [o.a.c.s.s.XenserverSnapshotStrategy] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) Snapshot: 2 doesn't have children, so it's ok to delete it and its parents
> 2014-10-28 15:16:42,347 DEBUG [c.c.a.t.Request] (API-Job-Executor-30:ctx-0b9151e2 job-53
ctx-3bc74f24) Seq 4-4574812796478292951: Sending  { Cmd , MgmtId: 6637838401571, via: 4(s-2-QA),
Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/4/7/ecb858aa-1656-4594-996a-869f1695bce2/ecb858aa-1656-4594-996a-869f1695bce2","volume":{"uuid":"5b639b64-a539-4651-ac4e-f06b69f37d97","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"adc69295-88b2-3db8-9e55-fe7db3847af9","id":1,"poolType":"NetworkFilesystem","host":"10.147.28.7","path":"/export/home/sanjeev/pri_cluster","port":2049,"url":"NetworkFilesystem://10.147.28.7/export/home/sanjeev/pri_cluster/?ROLE=Primary&STOREUUID=adc69295-88b2-3db8-9e55-fe7db3847af9"}},"name":"ROOT-7","size":2147483648,"path":"ROOT-7","volumeId":7,"vmName":"i-4-7-QA","accountId":4,"chainInfo":"{\"diskDeviceBusName\":\"ide0:1\",\"diskChain\":[\"[adc6929588b23db89e55fe7db3847af9]
i-4-7-QA/ROOT-7.vmdk\",\"[adc6929588b23db89e55fe7db3847af9] 0bc482b12d62345e9fca3cbce554086c/0bc482b12d62345e9fca3cbce554086c.vmdk\"]}","format":"OVA","provisioningType":"THIN","id":7,"deviceId":0,"hypervisorType":"VMware"},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/sanjeev/sec_45","_role":"Image"}},"vmName":"i-4-7-QA","name":"v2_ROOT-7_20141027213212","hypervisorType":"VMware","id":2,"quiescevm":false,"physicalSize":0}},"wait":0}}]
}
> 2014-10-28 15:16:42,359 DEBUG [c.c.a.t.Request] (AgentManager-Handler-16:null) Seq 4-4574812796478292951:
Processing:  { Ans: , MgmtId: 6637838401571, via: 4, Ver: v1, Flags: 10, [{"com.cloud.agent.api.Answer":{"result":true,"details":"snapshot
directory ecb858aa-1656-4594-996a-869f1695bce2 doesn't exist","wait":0}}] }
> 2014-10-28 15:16:42,360 DEBUG [c.c.a.t.Request] (API-Job-Executor-30:ctx-0b9151e2 job-53
ctx-3bc74f24) Seq 4-4574812796478292951: Received:  { Ans: , MgmtId: 6637838401571, via: 4,
Ver: v1, Flags: 10, { Answer } }
> 2014-10-28 15:16:42,434 DEBUG [c.c.a.t.Request] (API-Job-Executor-30:ctx-0b9151e2 job-53
ctx-3bc74f24) Seq 4-4574812796478292952: Sending  { Cmd , MgmtId: 6637838401571, via: 4(s-2-QA),
Ver: v1, Flags: 100111, [{"com.cloud.agent.api.DeleteSnapshotsDirCommand":{"store":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/sanjeev/sec_45","_role":"Image"}},"directory":"snapshots/4/8","wait":0}}]
}
> 2014-10-28 15:16:43,338 DEBUG [c.c.a.t.Request] (AgentManager-Handler-19:null) Seq 4-4574812796478292952:
Processing:  { Ans: , MgmtId: 6637838401571, via: 4, Ver: v1, Flags: 110, [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}]
}
> 2014-10-28 15:16:43,339 DEBUG [c.c.a.t.Request] (API-Job-Executor-30:ctx-0b9151e2 job-53
ctx-3bc74f24) Seq 4-4574812796478292952: Received:  { Ans: , MgmtId: 6637838401571, via: 4,
Ver: v1, Flags: 110, { Answer } }
> 2014-10-28 15:16:43,339 DEBUG [c.c.s.s.SnapshotManagerImpl] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) Deleted all snapshots for volume: 8 under account: 4
> 2014-10-28 15:16:43,342 DEBUG [c.c.a.m.AgentAttache] (AgentManager-Handler-19:null) Seq
4-4574812796478292952: No more commands found
> 2014-10-28 15:16:43,354 DEBUG [o.a.c.s.s.XenserverSnapshotStrategy] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) delete snapshot chain for snapshot: 3
> 2014-10-28 15:16:43,356 DEBUG [o.a.c.s.s.XenserverSnapshotStrategy] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) Snapshot: 3 doesn't have children, so it's ok to delete it and its parents
> 2014-10-28 15:16:43,389 DEBUG [c.c.a.t.Request] (API-Job-Executor-30:ctx-0b9151e2 job-53
ctx-3bc74f24) Seq 4-4574812796478292953: Sending  { Cmd , MgmtId: 6637838401571, via: 4(s-2-QA),
Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/4/8/657a763c-4fc6-49e0-8e7b-cb1ad2471a41/657a763c-4fc6-49e0-8e7b-cb1ad2471a41","volume":{"uuid":"1aebff64-f4c6-45bb-add1-249d54eca270","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"adc69295-88b2-3db8-9e55-fe7db3847af9","id":1,"poolType":"NetworkFilesystem","host":"10.147.28.7","path":"/export/home/sanjeev/pri_cluster","port":2049,"url":"NetworkFilesystem://10.147.28.7/export/home/sanjeev/pri_cluster/?ROLE=Primary&STOREUUID=adc69295-88b2-3db8-9e55-fe7db3847af9"}},"name":"ROOT-8","size":2147483648,"path":"ROOT-8","volumeId":8,"vmName":"i-4-8-QA","accountId":4,"chainInfo":"{\"diskDeviceBusName\":\"ide0:1\",\"diskChain\":[\"[adc6929588b23db89e55fe7db3847af9]
i-4-8-QA/ROOT-8.vmdk\",\"[adc6929588b23db89e55fe7db3847af9] 0bc482b12d62345e9fca3cbce554086c/0bc482b12d62345e9fca3cbce554086c.vmdk\"]}","format":"OVA","provisioningType":"THIN","id":8,"deviceId":0,"hypervisorType":"VMware"},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/sanjeev/sec_45","_role":"Image"}},"vmName":"i-4-8-QA","name":"v3_ROOT-8_20141027213237","hypervisorType":"VMware","id":3,"quiescevm":false,"physicalSize":0}},"wait":0}}]
}
> 2014-10-28 15:16:43,400 DEBUG [c.c.a.t.Request] (AgentManager-Handler-20:null) Seq 4-4574812796478292953:
Processing:  { Ans: , MgmtId: 6637838401571, via: 4, Ver: v1, Flags: 10, [{"com.cloud.agent.api.Answer":{"result":true,"details":"snapshot
directory 657a763c-4fc6-49e0-8e7b-cb1ad2471a41 doesn't exist","wait":0}}] }
> 2014-10-28 15:16:43,400 DEBUG [c.c.a.t.Request] (API-Job-Executor-30:ctx-0b9151e2 job-53
ctx-3bc74f24) Seq 4-4574812796478292953: Received:  { Ans: , MgmtId: 6637838401571, via: 4,
Ver: v1, Flags: 10, { Answer } }
> 2014-10-28 15:16:43,483 DEBUG [c.c.a.t.Request] (API-Job-Executor-30:ctx-0b9151e2 job-53
ctx-3bc74f24) Seq 4-4574812796478292954: Sending  { Cmd , MgmtId: 6637838401571, via: 4(s-2-QA),
Ver: v1, Flags: 100111, [{"com.cloud.agent.api.DeleteSnapshotsDirCommand":{"store":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/sanjeev/sec_45","_role":"Image"}},"directory":"snapshots/4/9","wait":0}}]
}
> 2014-10-28 15:16:43,758 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-6ecc0a29)
AutoScaling Monitor is running...
> 2014-10-28 15:16:44,555 DEBUG [c.c.a.t.Request] (AgentManager-Handler-2:null) Seq 4-4574812796478292954:
Processing:  { Ans: , MgmtId: 6637838401571, via: 4, Ver: v1, Flags: 110, [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}]
}
> 2014-10-28 15:16:44,556 DEBUG [c.c.a.t.Request] (API-Job-Executor-30:ctx-0b9151e2 job-53
ctx-3bc74f24) Seq 4-4574812796478292954: Received:  { Ans: , MgmtId: 6637838401571, via: 4,
Ver: v1, Flags: 110, { Answer } }
> 2014-10-28 15:16:44,556 DEBUG [c.c.s.s.SnapshotManagerImpl] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) Deleted all snapshots for volume: 9 under account: 4
> 2014-10-28 15:16:44,559 DEBUG [c.c.a.m.AgentAttache] (AgentManager-Handler-2:null) Seq
4-4574812796478292954: No more commands found
> 2014-10-28 15:16:44,579 DEBUG [o.a.c.s.s.XenserverSnapshotStrategy] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) delete snapshot chain for snapshot: 4
> 2014-10-28 15:16:44,582 DEBUG [o.a.c.s.s.XenserverSnapshotStrategy] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) Snapshot: 4 doesn't have children, so it's ok to delete it and its parents
> 2014-10-28 15:16:44,616 DEBUG [c.c.a.t.Request] (API-Job-Executor-30:ctx-0b9151e2 job-53
ctx-3bc74f24) Seq 4-4574812796478292955: Sending  { Cmd , MgmtId: 6637838401571, via: 4(s-2-QA),
Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/4/9/e7d15b83-06be-4596-9572-09478415e776/e7d15b83-06be-4596-9572-09478415e776","volume":{"uuid":"e88d9c15-e81b-44fd-99be-4d14f3d19c06","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"adc69295-88b2-3db8-9e55-fe7db3847af9","id":1,"poolType":"NetworkFilesystem","host":"10.147.28.7","path":"/export/home/sanjeev/pri_cluster","port":2049,"url":"NetworkFilesystem://10.147.28.7/export/home/sanjeev/pri_cluster/?ROLE=Primary&STOREUUID=adc69295-88b2-3db8-9e55-fe7db3847af9"}},"name":"ROOT-9","size":2147483648,"path":"ROOT-9","volumeId":9,"vmName":"i-4-9-QA","accountId":4,"chainInfo":"{\"diskDeviceBusName\":\"ide0:1\",\"diskChain\":[\"[adc6929588b23db89e55fe7db3847af9]
i-4-9-QA/ROOT-9.vmdk\",\"[adc6929588b23db89e55fe7db3847af9] 0bc482b12d62345e9fca3cbce554086c/0bc482b12d62345e9fca3cbce554086c.vmdk\"]}","format":"OVA","provisioningType":"THIN","id":9,"deviceId":0,"hypervisorType":"VMware"},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/sanjeev/sec_45","_role":"Image"}},"vmName":"i-4-9-QA","name":"v4_ROOT-9_20141027213330","hypervisorType":"VMware","id":4,"quiescevm":false,"physicalSize":0}},"wait":0}}]
}
> 2014-10-28 15:16:44,628 DEBUG [c.c.a.t.Request] (AgentManager-Handler-1:null) Seq 4-4574812796478292955:
Processing:  { Ans: , MgmtId: 6637838401571, via: 4, Ver: v1, Flags: 10, [{"com.cloud.agent.api.Answer":{"result":true,"details":"snapshot
directory e7d15b83-06be-4596-9572-09478415e776 doesn't exist","wait":0}}] }
> 2014-10-28 15:16:44,628 DEBUG [c.c.a.t.Request] (API-Job-Executor-30:ctx-0b9151e2 job-53
ctx-3bc74f24) Seq 4-4574812796478292955: Received:  { Ans: , MgmtId: 6637838401571, via: 4,
Ver: v1, Flags: 10, { Answer } }
> 2014-10-28 15:16:44,692 DEBUG [c.c.a.t.Request] (API-Job-Executor-30:ctx-0b9151e2 job-53
ctx-3bc74f24) Seq 4-4574812796478292956: Sending  { Cmd , MgmtId: 6637838401571, via: 4(s-2-QA),
Ver: v1, Flags: 100111, [{"com.cloud.agent.api.DeleteSnapshotsDirCommand":{"store":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/sanjeev/sec_45","_role":"Image"}},"directory":"snapshots/4/10","wait":0}}]
}
> 2014-10-28 15:16:44,805 DEBUG [c.c.a.t.Request] (AgentManager-Handler-3:null) Seq 4-4574812796478292956:
Processing:  { Ans: , MgmtId: 6637838401571, via: 4, Ver: v1, Flags: 110, [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}]
}
> 2014-10-28 15:16:44,805 DEBUG [c.c.a.m.AgentAttache] (AgentManager-Handler-3:null) Seq
4-4574812796478292956: No more commands found
> 2014-10-28 15:16:44,805 DEBUG [c.c.a.t.Request] (API-Job-Executor-30:ctx-0b9151e2 job-53
ctx-3bc74f24) Seq 4-4574812796478292956: Received:  { Ans: , MgmtId: 6637838401571, via: 4,
Ver: v1, Flags: 110, { Answer } }
> 2014-10-28 15:16:44,805 DEBUG [c.c.s.s.SnapshotManagerImpl] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) Deleted all snapshots for volume: 10 under account: 4
> 2014-10-28 15:16:44,820 DEBUG [o.a.c.s.s.XenserverSnapshotStrategy] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) delete snapshot chain for snapshot: 5
> 2014-10-28 15:16:44,822 DEBUG [o.a.c.s.s.XenserverSnapshotStrategy] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) Snapshot: 5 doesn't have children, so it's ok to delete it and its parents
> 2014-10-28 15:16:44,849 DEBUG [c.c.a.t.Request] (API-Job-Executor-30:ctx-0b9151e2 job-53
ctx-3bc74f24) Seq 4-4574812796478292957: Sending  { Cmd , MgmtId: 6637838401571, via: 4(s-2-QA),
Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/4/10/1c4b5c4a-b14e-4e42-9a0e-698215db758a/1c4b5c4a-b14e-4e42-9a0e-698215db758a","volume":{"uuid":"cb40dae3-6c47-40b1-8f07-f7ef6d095995","volumeType":"DATADISK","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"adc69295-88b2-3db8-9e55-fe7db3847af9","id":1,"poolType":"NetworkFilesystem","host":"10.147.28.7","path":"/export/home/sanjeev/pri_cluster","port":2049,"url":"NetworkFilesystem://10.147.28.7/export/home/sanjeev/pri_cluster/?ROLE=Primary&STOREUUID=adc69295-88b2-3db8-9e55-fe7db3847af9"}},"name":"Data-10","size":5368709120,"path":"9bd9c800f78545db87fc805e9f3fec1d","volumeId":10,"accountId":4,"format":"OVA","provisioningType":"THIN","id":10,"hypervisorType":"VMware"},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/sanjeev/sec_45","_role":"Image"}},"name":"detached_Data-10_20141027214535","hypervisorType":"VMware","id":5,"quiescevm":false,"physicalSize":0}},"wait":0}}]
}
> 2014-10-28 15:16:44,860 DEBUG [c.c.a.t.Request] (AgentManager-Handler-5:null) Seq 4-4574812796478292957:
Processing:  { Ans: , MgmtId: 6637838401571, via: 4, Ver: v1, Flags: 10, [{"com.cloud.agent.api.Answer":{"result":true,"details":"snapshot
directory 1c4b5c4a-b14e-4e42-9a0e-698215db758a doesn't exist","wait":0}}] }
> 2014-10-28 15:16:44,860 DEBUG [c.c.a.t.Request] (API-Job-Executor-30:ctx-0b9151e2 job-53
ctx-3bc74f24) Seq 4-4574812796478292957: Received:  { Ans: , MgmtId: 6637838401571, via: 4,
Ver: v1, Flags: 10, { Answer } }
> 2014-10-28 15:16:44,928 DEBUG [c.c.a.t.Request] (API-Job-Executor-30:ctx-0b9151e2 job-53
ctx-3bc74f24) Seq 4-4574812796478292958: Sending  { Cmd , MgmtId: 6637838401571, via: 4(s-2-QA),
Ver: v1, Flags: 100111, [{"com.cloud.agent.api.DeleteSnapshotsDirCommand":{"store":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/sanjeev/sec_45","_role":"Image"}},"directory":"snapshots/4/11","wait":0}}]
}
> 2014-10-28 15:16:44,975 DEBUG [c.c.a.t.Request] (AgentManager-Handler-4:null) Seq 4-4574812796478292958:
Processing:  { Ans: , MgmtId: 6637838401571, via: 4, Ver: v1, Flags: 110, [{"com.cloud.agent.api.Answer":{"result":true,"details":"snapshot
directory 11 doesn't exist","wait":0}}] }
> 2014-10-28 15:16:44,975 DEBUG [c.c.a.m.AgentAttache] (AgentManager-Handler-4:null) Seq
4-4574812796478292958: No more commands found
> 2014-10-28 15:16:44,975 DEBUG [c.c.a.t.Request] (API-Job-Executor-30:ctx-0b9151e2 job-53
ctx-3bc74f24) Seq 4-4574812796478292958: Received:  { Ans: , MgmtId: 6637838401571, via: 4,
Ver: v1, Flags: 110, { Answer } }
> 2014-10-28 15:16:44,975 DEBUG [c.c.s.s.SnapshotManagerImpl] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) Deleted all snapshots for volume: 11 under account: 4
> 2014-10-28 15:16:44,985 WARN  [c.c.u.AccountManagerImpl] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) Failed to cleanup account Acct[83f26103-9d83-4e4e-8bcc-c282c2acb498-acc]
due to
> com.cloud.exception.InvalidParameterValueException: Can't delete snapshotshot 6 due to
it is in Allocated Status
>         at org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.deleteSnapshot(XenserverSnapshotStrategy.java:213)
>         at com.cloud.storage.snapshot.SnapshotManagerImpl.deleteSnapshotDirsForAccount(SnapshotManagerImpl.java:623)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:601)
>         at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
>         at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
>         at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
>         at $Proxy180.deleteSnapshotDirsForAccount(Unknown Source)
>         at com.cloud.user.AccountManagerImpl.cleanupAccount(AccountManagerImpl.java:706)
>         at com.cloud.user.AccountManagerImpl.deleteAccount(AccountManagerImpl.java:665)
>         at com.cloud.user.AccountManagerImpl.deleteUserAccount(AccountManagerImpl.java:1444)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:601)
>         at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
>         at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:106)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
>         at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
>         at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
>         at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
>         at $Proxy103.deleteUserAccount(Unknown Source)
>         at org.apache.cloudstack.region.RegionManagerImpl.deleteUserAccount(RegionManagerImpl.java:187)
>         at org.apache.cloudstack.region.RegionServiceImpl.deleteUserAccount(RegionServiceImpl.java:121)
>         at org.apache.cloudstack.api.command.admin.account.DeleteAccountCmd.execute(DeleteAccountCmd.java:104)
>         at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
>         at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
>         at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:513)
>         at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>         at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>         at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:470)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:722)
> 2014-10-28 15:16:45,008 INFO  [c.c.u.AccountManagerImpl] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) Cleanup for account 4 is needed.
> 2014-10-28 15:16:45,047 DEBUG [c.c.a.ApiServlet] (catalina-exec-24:ctx-bcb44ca9) ===START===
 10.252.193.8 -- GET  command=queryAsyncJobResult&jobId=2c5b0fa0-ef16-4a61-bd64-7cae6acfdaa4&response=json&sessionkey=EpI1EuP5ZFF0VKispQyNb9AxWF4%3D&_=1414470673593
> 2014-10-28 15:16:45,082 DEBUG [c.c.a.ApiServlet] (catalina-exec-24:ctx-bcb44ca9 ctx-82b3efc5)
===END===  10.252.193.8 -- GET  command=queryAsyncJobResult&jobId=2c5b0fa0-ef16-4a61-bd64-7cae6acfdaa4&response=json&sessionkey=EpI1EuP5ZFF0VKispQyNb9AxWF4%3D&_=1414470673593
> 2014-10-28 15:16:45,230 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) Complete async job-53, jobStatus: SUCCEEDED, resultCode: 0, result: org.apache.cloudstack.api.response.SuccessResponse/null/{"success":true}
> 2014-10-28 15:16:45,232 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) Publish async job-53 complete on message bus
> 2014-10-28 15:16:45,233 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) Wake up jobs related to job- 53
> 2014-10-28 15:16:45,233 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) Update db status for job- 53
> 2014-10-28 15:16:45,236 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-30:ctx-0b9151e2
job-53 ctx-3bc74f24) Wake up jobs joined with job- 53 and disjoin all subjobs created from
job- 53
> 2014-10-28 15:16:45,241 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-30:ctx-0b9151e2
job-53) Done executing org.apache.cloudstack.api.command.admin.account.DeleteAccountCmd for
job-53
> 2014-10-28 15:16:45,248 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (API-Job-Executor-30:ctx-0b9151e2
job-53) Remove job-53 from job monitoring
> Even the account cleanup thread could not clean any of the remained artifacts like vms,networks



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message