incubator-cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alena Prokharchyk (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CLOUDSTACK-1368) Shared network - Not able to delete network because of java.lang.NullPointerException.
Date Wed, 06 Mar 2013 00:22:13 GMT

    [ https://issues.apache.org/jira/browse/CLOUDSTACK-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13594142#comment-13594142
] 

Alena Prokharchyk commented on CLOUDSTACK-1368:
-----------------------------------------------

It is a big bug in Network State transitioning introduced with the commit e7a554fc6a23a49949c2d88d6ef680682c6f6bc4
(Events framework merge). Murali, reassigning it to you. See the history of the problem and
the steps to fix the below. It should be fixed in both 4.1 and master branches.

History: 

* Before 4.1 although the Network state machine was in place (Network.java), it was never
referred in the code, All the state changes were hardcoded in the NetworkManagerImpl - wrong
way to do.

* After bringing up the Event manager code, all state changes for the network started going
through the state machine, which is totally correct and that's how it should have been implemented
from the very beginning. The problem is, the some state transitions are incorrect, and some
are missing. Therefore the incoming bugs.

Things to fix:

1) Add state transitions for the "Setup" state. That's what's causing this particular bug
to happen - all shared networks come in Setup state by default, and never go to Implemented
state.

2)  Change event for the line: 

s_fsm.addTransition(State.Implemented, Event.DestroyNetwork, State.Shutdown)

The event should be Event.Shutdown instead of Event.DestroyNetwork. 

3) Add state transition for Destroy event. The original state should be "Shutdown", the target
state in success case - 'Destroy', in failure case - 'Shutdown'

4)  Fix the comment for the following line (Shutdown states for the network being Shutdown,
not Destroy. Shutdown call can be triggered either by network GC thread, or by Destroy network
call): 

"Shutdown("Indicates the network configuration is being destroyed")"

5) Go through all the methods propagating the state change events in NetworkManagerImpl and
see if the proper events are being passed in. "Shutdown" event should be passed from shutdown
call, and "Destroy" event should be passed only from destroy network.

6) I know that you've added the Shutdown support for the Shared network, so they can be picked
up by the network GC. Make sure that state transitions are supported for this case (the network
comes in Setup state by default)

After putting in changes, QA for both Shared and Isolated network should be done. Have to
test the network lifecycle - create/shutdown (GC)/destroy networks of both types. Murali,
after unittest is passed, please come up with the list of tests for the QA so all the methods
where state machine use was introduced, are covered. Everything should behave as it used to
before the change.
                
> Shared network - Not able to delete network because of  java.lang.NullPointerException.
> ---------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-1368
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-1368
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Management Server
>    Affects Versions: 4.1.0
>         Environment: Latest build from 4.1
>            Reporter: Sangeetha Hariharan
>            Assignee: Alena Prokharchyk
>            Priority: Blocker
>             Fix For: 4.1.0
>
>
> Shared network - Not able to delete network because of  java.lang.NullPointerException.
> Set up:
> Advanced Zone with rhel 6.3 KVM host.
> Create a shared network with dual stack (ipv4 and ipv6).
> Delete this network.
> Delete netwpork fails with "Command failed due to Internal Server Error" errotr message:

> 2013-02-21 17:00:19,473 INFO  [cloud.api.ApiServer] (catalina-exec-20:null) (userId=2
accountId=2 sessionId=8323A1028B9235FCD786CDEAA7535425) 10.216.50.215 -- GET command=deleteNetwor
> k&id=b8ebcb5e-f0db-4997-9080-b7156d56bab9&response=json&sessionkey=LUODgzUxTfC7VuYYEVeNQ0oNo2c%3D&_=1361494821669
200 { "deletenetworkresponse" : {"jobid":"a2201560-eaf5-4d08-a5a7-afb
> d9e209934"} }
> 2013-02-21 17:00:22,671 INFO  [cloud.api.ApiServer] (catalina-exec-17:null) (userId=2
accountId=2 sessionId=8323A1028B9235FCD786CDEAA7535425) 10.216.50.215 -- GET command=queryAsyncJo
> bResult&jobId=a2201560-eaf5-4d08-a5a7-afbd9e209934&response=json&sessionkey=LUODgzUxTfC7VuYYEVeNQ0oNo2c%3D&_=1361494824961
200 { "queryasyncjobresultresponse" : {"accountid":"9bdc3f60
> -7c7d-11e2-89d9-06d4460004b1","userid":"9bdcb7c4-7c7d-11e2-89d9-06d4460004b1","cmd":"org.apache.cloudstack.api.command.user.network.DeleteNetworkCmd","jobstatus":2,"jobprocstatus":0,"
> jobresultcode":530,"jobresulttype":"object","jobresult":{"errorcode":530,"errortext":"Command
failed due to Internal Server Error"},"created":"2013-02-21T17:00:19-0800","jobid":"a2201
> 560-eaf5-4d08-a5a7-afbd9e209934"} }
> Management-server.logs:
> 2013-02-21 17:00:19,472 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-20:null)
submit async job-13, details
> : AsyncJobVO {id:13, userId: 2, accountId: 2, sessionKey: null, instanceType: null, instanceId:
null, cmd: org.apach
> e.cloudstack.api.command.user.network.DeleteNetworkCmd, cmdOriginator: null, cmdInfo:
{"id":"b8ebcb5e-f0db-4997-9080
> -b7156d56bab9","response":"json","sessionkey":"LUODgzUxTfC7VuYYEVeNQ0oNo2c\u003d","ctxUserId":"2","_":"1361494821669
> ","ctxAccountId":"2","ctxStartEventId":"45"}, cmdVersion: 0, callbackType: 0, callbackAddress:
null, status: 0, proc
> essStatus: 0, resultCode: 0, result: null, initMsid: 7508777239729, completeMsid: null,
lastUpdated: null, lastPolle
> d: null, created: null}
> 2013-02-21 17:00:19,473 DEBUG [cloud.api.ApiServlet] (catalina-exec-20:null) ===END===
 10.216.50.215 -- GET  comman
> d=deleteNetwork&id=b8ebcb5e-f0db-4997-9080-b7156d56bab9&response=json&sessionkey=LUODgzUxTfC7VuYYEVeNQ0oNo2c%3D&_=13
> 61494821669
> 2013-02-21 17:00:19,478 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-13:job-13)
Executing org.apache.clouds
> tack.api.command.user.network.DeleteNetworkCmd for job-13
> 2013-02-21 17:00:19,514 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-13:job-13)
Sync job-13 execution on ob
> ject network.205
> 2013-02-21 17:00:19,547 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-13:job-13)
job org.apache.cloudstack.a
> pi.command.user.network.DeleteNetworkCmd for job-13 was queued, processing the queue.
> 2013-02-21 17:00:19,555 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-13:job-13)
Executing sync queue item:
> SyncQueueItemVO {id:1, queueId: 1, contentType: AsyncJob, contentId: 13, lastProcessMsid:
7508777239729, lastprocess
> Number: 1, lastProcessTime: Thu Feb 21 17:00:19 PST 2013, created: Thu Feb 21 17:00:19
PST 2013}
> 2013-02-21 17:00:19,556 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-13:job-13)
Schedule queued job-13
> 2013-02-21 17:00:19,561 DEBUG [cloud.async.SyncQueueManagerImpl] (Job-Executor-13:job-13)
There is a pending process
> in sync queue(id: 1)
> 2013-02-21 17:00:19,562 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-14:job-13)
Executing org.apache.clouds
> tack.api.command.user.network.DeleteNetworkCmd for job-13
> 2013-02-21 17:00:19,607 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-14:job-13)
Network is not implemented
> : Ntwk[205|Guest|7]
> 2013-02-21 17:00:19,620 DEBUG [network.rules.RulesManagerImpl] (Job-Executor-14:job-13)
Releasing 0 port forwarding
> rules for network id=205
> 2013-02-21 17:00:19,621 DEBUG [network.rules.RulesManagerImpl] (Job-Executor-14:job-13)
Releasing 0 static nat rules
> for network id=205
> 2013-02-21 17:00:19,624 DEBUG [network.rules.RulesManagerImpl] (Job-Executor-14:job-13)
There are no port forwarding
> rules to apply for network id=205
> 2013-02-21 17:00:19,625 DEBUG [network.rules.RulesManagerImpl] (Job-Executor-14:job-13)
There are no static nat rule
> s to apply for network id=205
> 2013-02-21 17:00:19,627 DEBUG [network.rules.RulesManagerImpl] (Job-Executor-14:job-13)
Successfully released rules
> for network id=205 and # of rules now = 0
> 2013-02-21 17:00:19,627 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-14:job-13)
Successfully cleaned up po
> rtForwarding/staticNat rules for network id=205
> 2013-02-21 17:00:19,636 DEBUG [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-14:job-13)
Found 0 lb rules t
> o cleanup
> 2013-02-21 17:00:19,636 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-14:job-13)
Successfully cleaned up lo
> ad balancing rules for network id=205
> 2013-02-21 17:00:19,637 DEBUG [network.firewall.FirewallManagerImpl] (Job-Executor-14:job-13)
Releasing 0 firewall r
> ules for network id=205
> 2013-02-21 17:00:19,637 DEBUG [network.firewall.FirewallManagerImpl] (Job-Executor-14:job-13)
There are no firewall
> rules to apply
> 2013-02-21 17:00:19,638 DEBUG [network.firewall.FirewallManagerImpl] (Job-Executor-14:job-13)
Successfully released
> firewall rules for network id=205 and # of rules now = 0
> 2013-02-21 17:00:19,638 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-14:job-13)
Successfully cleaned up fi
> rewallRules rules for network id=205
> 2013-02-21 17:00:19,644 DEBUG [network.vpc.NetworkACLManagerImpl] (Job-Executor-14:job-13)
Found no network ACLs for
> network id=205
> 2013-02-21 17:00:19,644 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-14:job-13)
Successfully cleaned up Ne
> tworkACLs for network id=205
> 2013-02-21 17:00:19,652 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-14:job-13)
Sending destroy to com.clo
> ud.network.element.VirtualRouterElement@71fdbe17
> 2013-02-21 17:00:19,656 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-14:job-13)
Network id=205 is destroye
> d successfully, cleaning up corresponding resources now.
> 2013-02-21 17:00:19,673 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-14:job-13)
Deleted ip range for priva
> te network id=205
> 2013-02-21 17:00:19,673 DEBUG [db.Transaction.Transaction] (Job-Executor-14:job-13) Rolling
back the transaction: Ti
> me = 17 Name =  -AsyncJobManagerImpl$1.run:401-Executors$RunnableAdapter.call:471-FutureTask$Sync.innerRun:334-Futur
> eTask.run:166-ThreadPoolExecutor.runWorker:1110-ThreadPoolExecutor$Worker.run:603-Thread.run:679;
called by -Transac
> tion.rollback:890-Transaction.removeUpTo:833-Transaction.close:657-TransactionContextBuilder.AroundAnyMethod:39-Gene
> ratedMethodAccessor35.invoke:-1-DelegatingMethodAccessorImpl.invoke:43-Method.invoke:616-AbstractAspectJAdvice.invok
> eAdviceMethodWithGivenArgs:621-AbstractAspectJAdvice.invokeAdviceMethod:610-AspectJAroundAdvice.invoke:65-Reflective
> MethodInvocation.proceed:172-ExposeInvocationInterceptor.invoke:90
> 2013-02-21 17:00:19,714 ERROR [cloud.async.AsyncJobManagerImpl] (Job-Executor-14:job-13)
Unexpected exception while
> executing org.apache.cloudstack.api.command.user.network.DeleteNetworkCmd
> java.lang.NullPointerException
>         at com.cloud.utils.fsm.StateMachine2.getNextState(StateMachine2.java:81)
>         at com.cloud.utils.fsm.StateMachine2.transitTo(StateMachine2.java:100)
>         at com.cloud.network.NetworkManagerImpl.stateTransitTo(NetworkManagerImpl.java:3526)
>         at com.cloud.network.NetworkManagerImpl.destroyNetwork(NetworkManagerImpl.java:2246)
>         at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
>         at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.jav
> a:689)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
>         at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoi
> nPoint.java:80)
>         at com.cloud.utils.db.TransactionContextBuilder.AroundAnyMethod(TransactionContextBuilder.java:37)
>         at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:616)
>         at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
>         at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
>         at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
>         at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:90)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
>         at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:622)
>         at com.cloud.network.NetworkServiceImpl.deleteNetwork(NetworkServiceImpl.java:1343)
>         at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
>         at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:689)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
>         at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:80)
>         at com.cloud.event.ActionEventInterceptor.AroundAnyMethod(ActionEventInterceptor.java:41)
>         at sun.reflect.GeneratedMethodAccessor91.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:616)
>         at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
>         at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
>         at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
>         at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:80)
>         at com.cloud.utils.db.TransactionContextBuilder.AroundAnyMethod(TransactionContextBuilder.java:43)
>         at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:616)
>         at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
>         at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
>         at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message