cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CLOUDSTACK-5730) [Automation]DirectAgentAttache's send command fails due to ArrayIndexOutOfBoundsException; Unable to cleanup hypervisor's datastructures as part of Delete Account Job
Date Fri, 03 Jan 2014 10:19:54 GMT

    [ https://issues.apache.org/jira/browse/CLOUDSTACK-5730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13861395#comment-13861395
] 

ASF subversion and git services commented on CLOUDSTACK-5730:
-------------------------------------------------------------

Commit 6ce46e94f522f08c91db2cde33829f5b41539397 in branch refs/heads/4.3 from [~harikrishna.patnala]
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=6ce46e9 ]

CLOUDSTACK-5730: [Automation]DirectAgentAttache's send command fails due to ArrayIndexOutOfBoundsException;
Unable to cleanup hypervisor's datastructures as part of Delete Account Job

Signed-off-by: Koushik Das <koushik@apache.org>


> [Automation]DirectAgentAttache's send command fails due to ArrayIndexOutOfBoundsException;
Unable to cleanup hypervisor's datastructures as part of Delete Account Job
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-5730
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5730
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Management Server
>    Affects Versions: 4.3.0
>            Reporter: Chandan Purushothama
>            Assignee: Harikrishna Patnala
>            Priority: Critical
>             Fix For: 4.3.0
>
>
> =======================================================
> ArrayIndexOutOfBoundsException during DeleteAccount Job Execution:
> =======================================================
> 2014-01-03 07:39:16,752 DEBUG [c.c.v.VirtualMachineManagerImpl] (Job-Executor-72:ctx-d82a5609
ctx-1472cd68) Destroying vm VM[User|QA-88cfa2c8-e8ef-4133-8cf1-002ea3cb0c7b]
> 2014-01-03 07:39:16,752 DEBUG [c.c.v.VirtualMachineManagerImpl] (Job-Executor-72:ctx-d82a5609
ctx-1472cd68) Cleaning up NICS
> 2014-01-03 07:39:16,752 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Job-Executor-72:ctx-d82a5609
ctx-1472cd68) Cleaning network for vm: 44
> 2014-01-03 07:39:16,763 DEBUG [c.c.n.NetworkModelImpl] (Job-Executor-72:ctx-d82a5609
ctx-1472cd68) Service SecurityGroup is not supported in the network id=229
> 2014-01-03 07:39:16,780 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Job-Executor-72:ctx-d82a5609
ctx-1472cd68) Removed nic id=81
> 2014-01-03 07:39:16,782 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Job-Executor-72:ctx-d82a5609
ctx-1472cd68) Revoving nic secondary ip entry ...
> 2014-01-03 07:39:16,783 DEBUG [c.c.v.VirtualMachineManagerImpl] (Job-Executor-72:ctx-d82a5609
ctx-1472cd68) Cleaning up hypervisor data structures (ex. SRs in XenServer) for managed storage
> 2014-01-03 07:39:16,796 DEBUG [c.c.a.ApiServlet] (catalina-exec-5:ctx-f044cb4e) ===START===
 10.147.38.149 -- GET  signature=shTK1s%2FnQv84C1gwQYEUo%2F5WFW4%3D&apiKey=SyITk_W_BpbLs73aYJqNRBu9Sir1d7k6hqoKVUmFCMSVG-vshSHHKOdj_nVVTOQHqyvfR3ZSXhDzL2QUquSdJA&command=queryAsyncJobResult&response=json&jobid=fb908e1c-cdf8-4ae9-8c0d-09116061c3f7
> 2014-01-03 07:39:16,800 DEBUG [c.c.a.t.Request] (Job-Executor-72:ctx-d82a5609 ctx-1472cd68)
Seq 1-678298558: Sending  { Cmd , MgmtId: 6631563722783, via: 1(Rack1Pod1Host15), Ver: v1,
Flags: 100011, [] }
> 2014-01-03 07:39:16,801 DEBUG [c.c.a.t.Request] (Job-Executor-72:ctx-d82a5609 ctx-1472cd68)
Seq 1-678298558: Executing:  { Cmd , MgmtId: 6631563722783, via: 1(Rack1Pod1Host15), Ver:
v1, Flags: 100011, [] }
> 2014-01-03 07:39:16,801 WARN  [c.c.a.m.AgentAttache] (Job-Executor-72:ctx-d82a5609 ctx-1472cd68)
Seq 1-678298558: Unable to send due to
> java.lang.ArrayIndexOutOfBoundsException: 0
>         at com.cloud.agent.manager.DirectAgentAttache.send(DirectAgentAttache.java:100)
>         at com.cloud.agent.manager.AgentAttache.send(AgentAttache.java:374)
>         at com.cloud.agent.manager.AgentAttache.send(AgentAttache.java:398)
>         at com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:394)
>         at com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:920)
>         at com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineManagerImpl.java:497)
>         at com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineManagerImpl.java:448)
>         at com.cloud.vm.UserVmManagerImpl.expunge(UserVmManagerImpl.java:1704)
>         at sun.reflect.GeneratedMethodAccessor429.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:616)
>         at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
>         at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
>         at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
>         at $Proxy169.expunge(Unknown Source)
>         at com.cloud.user.AccountManagerImpl.cleanupAccount(AccountManagerImpl.java:629)
>         at com.cloud.user.AccountManagerImpl.deleteAccount(AccountManagerImpl.java:561)
>         at com.cloud.user.AccountManagerImpl.deleteUserAccount(AccountManagerImpl.java:1308)
>         at sun.reflect.GeneratedMethodAccessor478.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:616)
>         at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
>         at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:50)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
>         at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
>         at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
>         at $Proxy82.deleteUserAccount(Unknown Source)
>         at org.apache.cloudstack.region.RegionManagerImpl.deleteUserAccount(RegionManagerImpl.java:193)
>         at org.apache.cloudstack.region.RegionServiceImpl.deleteUserAccount(RegionServiceImpl.java:118)
>         at org.apache.cloudstack.api.command.admin.account.DeleteAccountCmd.execute(DeleteAccountCmd.java:101)
>         at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
>         at com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109)
>         at com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>         at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63)
>         at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:522)
>         at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>         at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2014-01-03 07:39:16,836 DEBUG [c.c.a.m.AgentAttache] (Job-Executor-72:ctx-d82a5609 ctx-1472cd68)
Seq 1-678298558: Cancelling.
> 2014-01-03 07:39:16,837 WARN  [c.c.v.UserVmManagerImpl] (Job-Executor-72:ctx-d82a5609
ctx-1472cd68) Unable to expunge  VM[User|QA-88cfa2c8-e8ef-4133-8cf1-002ea3cb0c7b]
> com.cloud.exception.AgentUnavailableException: Resource [Host:1] is unreachable: Host
1: Problem due to other exception 0
>         at com.cloud.agent.manager.AgentAttache.send(AgentAttache.java:389)
>         at com.cloud.agent.manager.AgentAttache.send(AgentAttache.java:398)
>         at com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:394)
>         at com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:920)
>         at com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineManagerImpl.java:497)
>         at com.cloud.vm.VirtualMachineManagerImpl.advanceExpunge(VirtualMachineManagerImpl.java:448)
>         at com.cloud.vm.UserVmManagerImpl.expunge(UserVmManagerImpl.java:1704)
>         at sun.reflect.GeneratedMethodAccessor429.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:616)
>         at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
>         at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
>         at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
>         at $Proxy169.expunge(Unknown Source)
>         at com.cloud.user.AccountManagerImpl.cleanupAccount(AccountManagerImpl.java:629)
>         at com.cloud.user.AccountManagerImpl.deleteAccount(AccountManagerImpl.java:561)
>         at com.cloud.user.AccountManagerImpl.deleteUserAccount(AccountManagerImpl.java:1308)
>         at sun.reflect.GeneratedMethodAccessor478.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:616)
>         at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
>         at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:50)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
>         at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
>         at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
>         at $Proxy82.deleteUserAccount(Unknown Source)
>         at org.apache.cloudstack.region.RegionManagerImpl.deleteUserAccount(RegionManagerImpl.java:193)
>         at org.apache.cloudstack.region.RegionServiceImpl.deleteUserAccount(RegionServiceImpl.java:118)
>         at org.apache.cloudstack.api.command.admin.account.DeleteAccountCmd.execute(DeleteAccountCmd.java:101)
>         at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
>         at com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109)
>         at com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>         at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63)
>         at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:522)
>         at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>         at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2014-01-03 07:39:16,839 ERROR [c.c.u.AccountManagerImpl] (Job-Executor-72:ctx-d82a5609
ctx-1472cd68) Unable to expunge vm: 44
> 2014-01-03 07:39:16,869 INFO  [c.c.u.AccountManagerImpl] (Job-Executor-72:ctx-d82a5609
ctx-1472cd68) deleteAccount: Deleted 1 network groups for account 26
> 2014-01-03 07:39:16,870 INFO  [c.c.u.AccountManagerImpl] (Job-Executor-72:ctx-d82a5609
ctx-1472cd68) deleteAccount: Deleted 0 affinity groups for account 26
> 2014-01-03 07:39:16,871 DEBUG [c.c.u.AccountManagerImpl] (Job-Executor-72:ctx-d82a5609
ctx-1472cd68) Deleting networks for account 26
> 2014-01-03 07:39:16,884 DEBUG [c.c.a.ApiServlet] (catalina-exec-5:ctx-f044cb4e ctx-e5d3f936
ctx-efc2b479) ===END===  10.147.38.149 -- GET  signature=shTK1s%2FnQv84C1gwQYEUo%2F5WFW4%3D&apiKey=SyITk_W_BpbLs73aYJqNRBu9Sir1d7k6hqoKVUmFCMSVG-vshSHHKOdj_nVVTOQHqyvfR3ZSXhDzL2QUquSdJA&command=queryAsyncJobResult&response=json&jobid=fb908e1c-cdf8-4ae9-8c0d-09116061c3f7
> 2014-01-03 07:39:16,894 WARN  [o.a.c.e.o.NetworkOrchestrator] (Job-Executor-72:ctx-d82a5609
ctx-1472cd68) Can't delete the network, not all user vms are expunged. Vm VM[User|QA-03d9e9fa-081a-4175-9bf2-2508034b6e17]
is in Expunging state
> 2014-01-03 07:39:16,894 WARN  [c.c.u.AccountManagerImpl] (Job-Executor-72:ctx-d82a5609
ctx-1472cd68) Unable to destroy network Ntwk[229|Guest|8] as a part of account id=26 cleanup.
> 2014-01-03 07:39:16,895 DEBUG [c.c.u.AccountManagerImpl] (Job-Executor-72:ctx-d82a5609
ctx-1472cd68) Deleting vpcs for account 26
> 2014-01-03 07:39:16,897 DEBUG [c.c.u.AccountManagerImpl] (Job-Executor-72:ctx-d82a5609
ctx-1472cd68) Deleting site-to-site VPN customer gateways for account 26
> 2014-01-03 07:39:16,908 INFO  [c.c.u.AccountManagerImpl] (Job-Executor-72:ctx-d82a5609
ctx-1472cd68) deleteAccount: Released 0 dedicated guest vlan ranges from account 26
> 2014-01-03 07:39:17,034 INFO  [c.c.u.AccountManagerImpl] (Job-Executor-72:ctx-d82a5609
ctx-1472cd68) Cleanup for account 26 is needed.
> 2014-01-03 07:39:17,048 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-72:ctx-d82a5609
ctx-1472cd68) Complete async job-359, jobStatus: SUCCEEDED, resultCode: 0, result: org.apache.cloudstack.api.response.SuccessResponse/null/{"success":true}
> 2014-01-03 07:39:17,056 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-72:ctx-d82a5609)
Done executing org.apache.cloudstack.api.command.admin.account.DeleteAccountCmd for job-359



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message