incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alena Prokharchyk (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CLOUDSTACK-279) Deleting project fails to delete any available VR
Date Mon, 08 Oct 2012 21:20:03 GMT

     [ https://issues.apache.org/jira/browse/CLOUDSTACK-279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alena Prokharchyk updated CLOUDSTACK-279:
-----------------------------------------


Just did more code analysis. The problem is the following:

1) deleteProject call can be executed by the regular user.
2) as a part of the call deleteAccount/cleanupAccount call is being made. Cleanup account
consists of 2 parts:

* account is marked as Removed in the DB
* then cleanup part starts up. And although System user/account are being passed to the cleanupAccount
method call, it doesn't pass the caller to all the methods called from within (in fact, some
of the methods miss "Account caller" in the signature). So deleteVolume/deleteRouter calls
retrieve the caller from the UserContext. And the userContext is a regular user, and he is
not supposed to cleanup the resources belonging to Removed project account.


As this problem was in the code from the very beginning, and fixing it would affect too many
places in the code, punting the bug to 4.1. There are workarounds for the bug:

- the account cleanup thread will eventually complete the project deletion
- ROOT/Domain admin can always execute the call on the regular user behalf; and in this case
the deletion will be successful. 
                
> Deleting project fails to delete any available VR
> -------------------------------------------------
>
>                 Key: CLOUDSTACK-279
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-279
>             Project: CloudStack
>          Issue Type: Bug
>          Components: Network Controller
>    Affects Versions: pre-4.0.0
>            Reporter: Rohit Yadav
>            Assignee: Alena Prokharchyk
>             Fix For: 4.0.0
>
>
> Steps to reproduce :
> -------------------------
> 1. Deploy a CS advanced networking setup.
> 2. create a sub-domain 'Domain1' under 'ROOT' domain.
> 3. Create 2 users - user1 and user2 under 'Domain1'.
> 4. login as user1 and create a project. Add user2 to the project.
> 5. Create 2 VMs , one as user1 and the other as user2.
> 6. Login as user1(project admin) and delete the project. 
> Tried against build http://jenkins.cloudstack.org/job/build-4.0-rhel63/437/artifact/CloudStack-oss-4.0.0-437.tar.bz2
and these were the observations :
> 1. This time there was no NPE while executing listRouter command. Refer: https://issues.apache.org/jira/browse/CLOUDSTACK-84
> 2. But the exception while deleting VR was still there
> 2012-10-06 20:27:03,531 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-16:job-16)
Unassiging ip address Ip[10.102.125.68-1]
> 2012-10-06 20:27:03,540 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-16:job-16)
Sending destroy to com.cloud.network.element.VirtualRouterElement$$EnhancerByCGLIB$$b3705199@6041646a
> 2012-10-06 20:27:03,542 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-16:job-16)
Attempting to destroy router 6
> 2012-10-06 20:27:03,546 WARN [cloud.network.NetworkManagerImpl] (Job-Executor-16:job-16)
Unable to complete destroy of the network due to element: VirtualRouter
> com.cloud.exception.PermissionDeniedException: Acct[3-user1] does not have permission
to operate with resource VM[DomainRouter|r-6-VM]
>         at com.cloud.acl.DomainChecker.checkAccess(DomainChecker.java:128)
>         at com.cloud.user.AccountManagerImpl.checkAccess(AccountManagerImpl.java:365)
>         at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.destroyRouter(VirtualNetworkApplianceManagerImpl.java:381)
>         at com.cloud.network.element.VirtualRouterElement.destroy(VirtualRouterElement.java:641)
>         at com.cloud.network.NetworkManagerImpl.destroyNetwork(NetworkManagerImpl.java:3554)
>         at com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
>         at com.cloud.user.AccountManagerImpl.cleanupAccount(AccountManagerImpl.java:618)
>         at com.cloud.user.AccountManagerImpl.deleteAccount(AccountManagerImpl.java:506)
>         at com.cloud.projects.ProjectManagerImpl.cleanupProject(ProjectManagerImpl.java:305)
>         at com.cloud.projects.ProjectManagerImpl.deleteProject(ProjectManagerImpl.java:286)
>         at com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
>         at com.cloud.projects.ProjectManagerImpl.deleteProject(ProjectManagerImpl.java:265)
>         at com.cloud.event.ActionEventCallback.intercept(ActionEventCallback.java:36)
>         at com.cloud.api.commands.DeleteProjectCmd.execute(DeleteProjectCmd.java:69)
>         at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
>         at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2012-10-06 20:27:03,547 WARN [cloud.user.AccountManagerImpl] (Job-Executor-16:job-16)
Unable to destroy network Ntwk[205|Guest|8] as a part of account id=5 cleanup.
> 2012-10-06 20:27:03,548 DEBUG [cloud.user.AccountManagerImpl] (Job-Executor-16:job-16)
Deleting vpcs for account 5
> 2012-10-06 20:27:03,550 DEBUG [cloud.user.AccountManagerImpl] (Job-Executor-16:job-16)
Deleting site-to-site VPN customer gateways for account 52012-10-06 20:27:03,569 INFO [cloud.user.AccountManagerImpl]
(Job-Executor-16:job-16) Cleanup for account 5 is needed.
> 3. In the above exception ,last line we see that cleanup for account 5 is needed. So
i changed the global settings "account.cleanup.interval" to 30s and restarted management server.
> After that I see that the VR is expunged when account cleanup was done. Is this the expected
behaviour?
> 2012-10-06 20:43:52,978 INFO [cloud.user.AccountManagerImpl] (AccountChecker-1:null)
Found 1 removed accounts to cleanup
> 2012-10-06 20:43:52,978 DEBUG [cloud.user.AccountManagerImpl] (AccountChecker-1:null)
Cleaning up 5
> 2012-10-06 20:4
> 2012-10-06 20:43:53,364 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (AccountChecker-1:null)
Attempting to destroy router 6
> 2012-10-06 20:43:53,369 DEBUG [cloud.vm.VirtualMachineManagerImpl] (AccountChecker-1:null)
VM is already stopped: VM[DomainRouter|r-6-VM]
> 2012-10-06 20:43:53,374 DEBUG [cloud.capacity.CapacityManagerImpl] (AccountChecker-1:null)
VM state transitted from :Stopped to Expunging with event: ExpungeOperationvm's original host
id: 1 new host id: null host id before state transition: null
> 2012-10-06 20:43:53,391 DEBUG [cloud.capacity.CapacityManagerImpl] (AccountChecker-1:null)
Hosts's actual total CPU: 17600 and CPU after applying overprovisioning: 17600
> 2012-10-06 20:43:53,391 DEBUG [cloud.capacity.CapacityManagerImpl] (AccountChecker-1:null)
release cpu from host: 1, old used: 2500,reserved: 500, actual total: 17600, total with overprovisioning:
17600; new used: 2500,reserved:0; movedfromreserved: true,moveToReserveredfalse
> 2012-10-06 20:43:53,392 DEBUG [cloud.capacity.CapacityManagerImpl] (AccountChecker-1:null)
release mem from host: 1, old used: 2550136832,reserved: 134217728, total: 33584095232; new
used: 2550136832,reserved:0; movedfromreserved: true,moveToReserveredfalse
> 2012-10-06 20:43:53,403 DEBUG [cloud.vm.VirtualMachineManagerImpl] (AccountChecker-1:null)
Destroying vm VM[DomainRouter|r-6-VM]
> 2012-10-06 20:43:53,403 DEBUG [cloud.vm.VirtualMachineManagerImpl] (AccountChecker-1:null)
Cleaning up NICS
> 2012-10-06 20:43:53,403 DEBUG [cloud.network.NetworkManagerImpl] (AccountChecker-1:null)
Cleaning network for vm: 6
> 2012-10-06 20:43:53,456 DEBUG [cloud.network.NetworkManagerImpl] (AccountChecker-1:null)
Service SecurityGroup is not supported in the network id=205
> 2012-10-06 20:43:53,492 DEBUG [cloud.network.NetworkManagerImpl] (AccountChecker-1:null)
Removed nic id=13
> 2012-10-06 20:43:53,516 DEBUG [cloud.network.NetworkManagerImpl] (AccountChecker-1:null)
Removed nic id=14
> 2012-10-06 20:43:53,530 DEBUG [network.guru.PublicNetworkGuru] (AccountChecker-1:null)
public network deallocate network: networkId: 200, ip: 10.102.125.68
> 2012-10-06 20:43:53,532 DEBUG [network.guru.PublicNetworkGuru] (AccountChecker-1:null)
Deallocated nic: NicProfile[15-6-null-null-null
> 2012-10-06 20:43:53,537 DEBUG [cloud.network.NetworkManagerImpl] (AccountChecker-1:null)
Removed nic id=15
> 2012-10-06 20:43:53,548 DEBUG [cloud.storage.StorageManagerImpl] (AccountChecker-1:null)
Cleaning storage for vm: 6
> 2012-10-06 20:43:53,592 DEBUG [cloud.storage.StorageManagerImpl] (AccountChecker-1:null)
Expunging Vol[6|vm=6|ROOT]
> 2012-10-06 20:43:53,592 DEBUG [cloud.storage.StorageManagerImpl] (AccountChecker-1:null)
Expunging Vol[6|vm=6|ROOT]
> 2012-10-06 20:43:53,615 DEBUG [agent.transport.Request] (AccountChecker-1:null) Seq 1-1625948170:
Sending { Cmd , MgmtId: 205209135696933, via: 1, Ver: v1, Flags: 100111, [{"storage.DestroyCommand":{"vmName":"r-6-VM","volume":{"id":6,"name":"ROOT-6","mountPoint":"/cloudstack/abhinav/primary","path":"daccc1a1-3768-4d79-886f-bf1b15423c46","size":725811200,"type":"ROOT","storagePoolType":"NetworkFilesystem","storagePoolUuid":"b5bf91a6-a41c-3c83-8a03-9775d0538865","deviceId":0},"wait":0}}]
}
> 2012-10-06 20:43:54,244 DEBUG [agent.transport.Request] (AgentManager-Handler-1:null)
Seq 1-1625948170: Processing: { Ans: , MgmtId: 205209135696933, via: 1, Ver: v1, Flags: 110,
[{"Answer":{"result":true,"details":"Success","wait":0}}] }
> 2012-10-06 20:43:54,244 DEBUG [agent.transport.Request] (AccountChecker-1:null) Seq 1-1625948170:
Received: { Ans: , MgmtId: 205209135696933, via: 1, Ver: v1, Flags: 110, { Answer } }
> 2012-10-06 20:43:54,249 DEBUG [agent.manager.AgentAttache] (AgentManager-Handler-1:null)
Seq 1-1625948170: No more commands found
> 2012-10-06 20:43:54,261 DEBUG [cloud.storage.StorageManagerImpl] (AccountChecker-1:null)
Volume successfully expunged from 200
> 2012-10-06 20:43:54,261 DEBUG [cloud.vm.VirtualMachineManagerImpl] (AccountChecker-1:null)
Expunged VM[DomainRouter|r-6-VM]
> 2012-10-06 20:43:54,274 DEBUG [cloud.network.NetworkManagerImpl] (AccountChecker-1:null)
Network id=205 is destroyed successfully, cleaning up corresponding resources now.
> 2012-10-06 20:43:54,297 DEBUG [cloud.network.NetworkManagerImpl] (AccountChecker-1:null)
Deleted ip range for private network id=205
> 2012-10-06 20:43:54,336 DEBUG [cloud.user.AccountManagerImpl] (AccountChecker-1:null)
Network 205 successfully deleted as a part of account id=5 cleanup.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message