cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daan Hoogland <daan.hoogl...@gmail.com>
Subject Re: unable to remove NIC from instance
Date Tue, 24 Feb 2015 07:51:58 GMT
Ok, thanks, we are running a custom 4.4.2 version we had to make due
to problems with the stock 4.4.2. I will look into making another
release candidate for 4.4.3 and have a look at the code to see if I
can find a cause to your problem. The last 4.4.3 release candidate I
made was largely ignored and therefore never released. It also has a
problem with creating new volumes on vmware which is not solved yet.
What type of hypervisors are you running?

On Tue, Feb 24, 2015 at 8:45 AM, Sonali Jadhav
<sonali@servercentralen.se> wrote:
> Hi Daan,
>
> Super thanks for reply. I am running version 4.4.2
>
> /Sonali
>
> -----Original Message-----
> From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> Sent: Tuesday, February 24, 2015 1:05 PM
> To: users@cloudstack.apache.org
> Subject: Re: unable to remove NIC from instance
>
> It seems to me you are a victim of overzealous exception handling.
> I've seen that happen in the async job handling. The job dispatching or construction
has an exception that is not handled but caught and ignored, it end and the async job is not
removed from the queue. I'd like to look at the code near the stack trace but for that I need
the exact version of Cloudstack you are running.
>
> On Tue, Feb 24, 2015 at 5:44 AM, Sonali Jadhav <sonali@servercentralen.se> wrote:
>> Hi Daan,
>>
>> I have checked logs form starting of api call, and I have not found
>> anything special, But apart from error, I found one warning,
>>
>> 2015-02-23 13:11:46,226 WARN  [c.c.u.d.Merovingian2]
>> (API-Job-Executor-118:ctx-adab6031 job-526 ctx-0585e904) Was unable to
>> find lock for the key vm_instance24 and thread id 137380473
>>
>> Don’t know meaning of this.
>>
>> Here  are my logs,
>>
>> 2015-02-23 13:11:45,457 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgentCronJob-296:ctx-11de89f6)
Seq 5-8090716730571096067: Response Received:
>> 2015-02-23 13:11:45,458 DEBUG [c.c.a.t.Request]
>> (DirectAgentCronJob-296:ctx-11de89f6) Seq 5-8090716730571096067:
>> Processing:  { Ans: , MgmtId: 59778234354585, via: 5, Ver: v1, Flags:
>> 10,
>> [{"com.cloud.agent.api.ClusterVMMetaDataSyncAnswer":{"_clusterId":1,"_
>> vmMetaDatum":{"i-2-11-VM":"viridian:true;acpi:1;apic:true;pae:true;nx:
>> true","s-15-VM":"viridian:true;acpi:1;apic:true;pae:true;nx:true","r-6
>> -VM":"viridian:true;acpi:1;apic:true;pae:true;nx:true","i-2-5-VM":"vir
>> idian:true;acpi:1;apic:true;pae:true;nx:true;timeoffset:0","r-13-VM":"
>> viridian:true;acpi:1;apic:true;pae:true;nx:true","i-5-12-VM":"viridian
>> :true;acpi:1;apic:true;pae:true;nx:true;timeoffset:0","r-14-VM":"virid
>> ian:true;acpi:1;apic:true;pae:true;nx:true","v-1-VM":"viridian:true;ac
>> pi:1;apic:true;pae:true;nx:true","i-9-23-VM":"viridian:true;acpi:1;api
>> c:true;pae:true;nx:true;timeoffset:-28800"},"_isExecuted":false,"resul
>> t":true,"wait":0}}] }
>> 2015-02-23 13:11:45,572 DEBUG [c.c.a.m.DirectAgentAttache]
>> (DirectAgent-48:ctx-9e4c61a9) Seq 1-7548877400403099459: Executing
>> request
>> 2015-02-23 13:11:45,628 DEBUG [c.c.h.x.r.CitrixResourceBase]
>> (DirectAgent-48:ctx-9e4c61a9) Vm cpu utilization 0.11499999999999999
>> 2015-02-23 13:11:45,650 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-48:ctx-9e4c61a9)
Seq 1-7548877400403099459: Response Received:
>> 2015-02-23 13:11:45,650 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
>> (DirectAgent-48:ctx-9e4c61a9) Seq 1-7548877400403099459: MgmtId
>> 249373720971888: Resp: Routing to peer
>> 2015-02-23 13:11:45,682 DEBUG [c.c.a.m.DirectAgentAttache]
>> (DirectAgent-95:ctx-5f7ee8a6) Seq 5-249105354388946804: Executing
>> request
>> 2015-02-23 13:11:45,707 DEBUG [c.c.h.x.r.CitrixResourceBase]
>> (DirectAgent-95:ctx-5f7ee8a6) Vm cpu utilization 0.1152941176470588
>> 2015-02-23 13:11:45,707 DEBUG [c.c.h.x.r.CitrixResourceBase]
>> (DirectAgent-95:ctx-5f7ee8a6) Vm cpu utilization 0.2599999999999999
>> 2015-02-23 13:11:45,766 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-95:ctx-5f7ee8a6)
Seq 5-249105354388946804: Response Received:
>> 2015-02-23 13:11:45,767 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
>> (DirectAgent-95:ctx-5f7ee8a6) Seq 5-249105354388946804: MgmtId
>> 249373720971888: Resp: Routing to peer
>> 2015-02-23 13:11:45,791 DEBUG [c.c.a.m.DirectAgentAttache]
>> (DirectAgent-192:ctx-61e81d25) Seq 6-2490209118959188220: Executing
>> request
>> 2015-02-23 13:11:45,835 DEBUG [c.c.h.x.r.CitrixResourceBase]
>> (DirectAgent-192:ctx-61e81d25) Vm cpu utilization 0.13343749999999996
>> 2015-02-23 13:11:45,858 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-192:ctx-61e81d25)
Seq 6-2490209118959188220: Response Received:
>> 2015-02-23 13:11:45,858 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
>> (DirectAgent-192:ctx-61e81d25) Seq 6-2490209118959188220: MgmtId
>> 249373720971888: Resp: Routing to peer
>> 2015-02-23 13:11:45,921 DEBUG [c.c.a.ApiServlet]
>> (http-6443-exec-4:ctx-5a612122) ===START===  192.168.45.32 -- GET
>> command=removeNicFromVirtualMachine&response=json&sessionkey=7O6X4iElw
>> 4iQSPf%2BX1ypVjWpwMo%3D&virtualmachineid=b5c535b4-10c4-44a9-b117-6df07
>> 537434c&nicid=99378e9f-ba64-4c76-ac16-d9ac977d979f&_=1424693402955
>> 2015-02-23 13:11:46,095 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (http-6443-exec-4:ctx-5a612122 ctx-b6192261) submit async job-526,
>> details: AsyncJobVO {id:526, userId: 2, accountId: 2, instanceType:
>> None, instanceId: null, cmd:
>> org.apache.cloudstack.api.command.admin.vm.RemoveNicFromVMCmdByAdmin,
>> cmdInfo:
>> {"response":"json","sessionkey":"7O6X4iElw4iQSPf+X1ypVjWpwMo\u003d","c
>> txDetails":"{\"com.cloud.vm.Nic\":\"99378e9f-ba64-4c76-ac16-d9ac977d97
>> 9f\",\"com.cloud.vm.VirtualMachine\":\"b5c535b4-10c4-44a9-b117-6df0753
>> 7434c\"}","virtualmachineid":"b5c535b4-10c4-44a9-b117-6df07537434c","c
>> mdEventType":"NIC.DELETE","nicid":"99378e9f-ba64-4c76-ac16-d9ac977d979
>> f","ctxUserId":"2","httpmethod":"GET","_":"1424693402955","ctxAccountI
>> d":"2","ctxStartEventId":"1160"}, cmdVersion: 0, status: IN_PROGRESS,
>> processStatus: 0, resultCode: 0, result: null, initMsid:
>> 59778234354585, completeMsid: null, lastUpdated: null, lastPolled:
>> null, created: null}
>> 2015-02-23 13:11:46,096 INFO  [o.a.c.f.j.i.AsyncJobMonitor]
>> (API-Job-Executor-118:ctx-adab6031 job-526) Add job-526 into job
>> monitoring
>> 2015-02-23 13:11:46,096 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-118:ctx-adab6031 job-526) Executing AsyncJobVO
>> {id:526, userId: 2, accountId: 2, instanceType: None, instanceId:
>> null, cmd:
>> org.apache.cloudstack.api.command.admin.vm.RemoveNicFromVMCmdByAdmin,
>> cmdInfo:
>> {"response":"json","sessionkey":"7O6X4iElw4iQSPf+X1ypVjWpwMo\u003d","c
>> txDetails":"{\"com.cloud.vm.Nic\":\"99378e9f-ba64-4c76-ac16-d9ac977d97
>> 9f\",\"com.cloud.vm.VirtualMachine\":\"b5c535b4-10c4-44a9-b117-6df0753
>> 7434c\"}","virtualmachineid":"b5c535b4-10c4-44a9-b117-6df07537434c","c
>> mdEventType":"NIC.DELETE","nicid":"99378e9f-ba64-4c76-ac16-d9ac977d979
>> f","ctxUserId":"2","httpmethod":"GET","_":"1424693402955","ctxAccountI
>> d":"2","ctxStartEventId":"1160"}, cmdVersion: 0, status: IN_PROGRESS,
>> processStatus: 0, resultCode: 0, result: null, initMsid:
>> 59778234354585, completeMsid: null, lastUpdated: null, lastPolled:
>> null, created: null}
>> 2015-02-23 13:11:46,097 DEBUG [c.c.a.ApiServlet]
>> (http-6443-exec-4:ctx-5a612122 ctx-b6192261) ===END===  192.168.45.32
>> -- GET
>> command=removeNicFromVirtualMachine&response=json&sessionkey=7O6X4iElw
>> 4iQSPf%2BX1ypVjWpwMo%3D&virtualmachineid=b5c535b4-10c4-44a9-b117-6df07
>> 537434c&nicid=99378e9f-ba64-4c76-ac16-d9ac977d979f&_=1424693402955
>> 2015-02-23 13:11:46,219 DEBUG [c.c.c.ConsoleProxyManagerImpl]
>> (consoleproxy-1:ctx-a5fe3580) Zone 1 is ready to launch console proxy
>> 2015-02-23 13:11:46,221 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-118:ctx-adab6031 job-526 ctx-0585e904) Sync job-527
>> execution on object VmWorkJobQueue.24
>> 2015-02-23 13:11:46,226 WARN  [c.c.u.d.Merovingian2]
>> (API-Job-Executor-118:ctx-adab6031 job-526 ctx-0585e904) Was unable to
>> find lock for the key vm_instance24 and thread id 137380473
>> 2015-02-23 13:11:46,534 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
>> (secstorage-1:ctx-be45d820) Zone 1 is ready to launch secondary
>> storage VM
>> 2015-02-23 13:11:49,095 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-4a027738)
HostStatsCollector is running...
>> 2015-02-23 13:11:49,115 DEBUG [c.c.a.m.DirectAgentAttache]
>> (DirectAgent-93:ctx-3810dbca) Seq 1-9127107594819755362: Executing
>> request
>> 2015-02-23 13:11:49,166 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-93:ctx-3810dbca)
Seq 1-9127107594819755362: Response Received:
>> 2015-02-23 13:11:49,166 DEBUG [c.c.a.t.Request]
>> (StatsCollector-4:ctx-4a027738) Seq 1-9127107594819755362: Received:
>> { Ans: , MgmtId: 59778234354585, via: 1, Ver: v1, Flags: 10, {
>> GetHostStatsAnswer } }
>> 2015-02-23 13:11:49,182 DEBUG [c.c.a.m.DirectAgentAttache]
>> (DirectAgent-4:ctx-5d8fe3d8) Seq 4-677791743919277703: Executing
>> request
>> 2015-02-23 13:11:49,226 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-4:ctx-5d8fe3d8)
Seq 4-677791743919277703: Response Received:
>> 2015-02-23 13:11:49,227 DEBUG [c.c.a.t.Request]
>> (StatsCollector-4:ctx-4a027738) Seq 4-677791743919277703: Received:  {
>> Ans: , MgmtId: 59778234354585, via: 4, Ver: v1, Flags: 10, {
>> GetHostStatsAnswer } }
>> 2015-02-23 13:11:49,242 DEBUG [c.c.a.m.DirectAgentAttache]
>> (DirectAgent-194:ctx-9a8ead36) Seq 5-8090716730571121376: Executing
>> request
>> 2015-02-23 13:11:49,280 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-194:ctx-9a8ead36)
Seq 5-8090716730571121376: Response Received:
>> 2015-02-23 13:11:49,280 DEBUG [c.c.a.t.Request]
>> (StatsCollector-4:ctx-4a027738) Seq 5-8090716730571121376: Received:
>> { Ans: , MgmtId: 59778234354585, via: 5, Ver: v1, Flags: 10, {
>> GetHostStatsAnswer } }
>> 2015-02-23 13:11:49,288 DEBUG [c.c.a.ApiServlet]
>> (http-6443-exec-3:ctx-c1a661f7) ===START===  192.168.45.32 -- GET
>> command=queryAsyncJobResult&jobId=af305f51-35d5-417a-acd7-ecb3ddadbb8a
>> &response=json&sessionkey=7O6X4iElw4iQSPf%2BX1ypVjWpwMo%3D&_=142469340
>> 6327
>> 2015-02-23 13:11:49,302 DEBUG [c.c.a.m.DirectAgentAttache]
>> (DirectAgent-383:ctx-5f562eca) Seq 6-892275676172801479: Executing
>> request
>> 2015-02-23 13:11:49,354 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-383:ctx-5f562eca)
Seq 6-892275676172801479: Response Received:
>> 2015-02-23 13:11:49,354 DEBUG [c.c.a.t.Request]
>> (StatsCollector-4:ctx-4a027738) Seq 6-892275676172801479: Received:  {
>> Ans: , MgmtId: 59778234354585, via: 6, Ver: v1, Flags: 10, {
>> GetHostStatsAnswer } }
>> 2015-02-23 13:11:49,626 DEBUG [c.c.a.ApiServlet]
>> (http-6443-exec-3:ctx-c1a661f7 ctx-62507b94) ===END===  192.168.45.32
>> -- GET
>> command=queryAsyncJobResult&jobId=af305f51-35d5-417a-acd7-ecb3ddadbb8a
>> &response=json&sessionkey=7O6X4iElw4iQSPf%2BX1ypVjWpwMo%3D&_=142469340
>> 6327
>> 2015-02-23 13:11:49,672 ERROR [c.c.a.ApiAsyncJobDispatcher]
>> (API-Job-Executor-118:ctx-adab6031 job-526) Unexpected exception while
>> executing
>> org.apache.cloudstack.api.command.admin.vm.RemoveNicFromVMCmdByAdmin
>> java.lang.RuntimeException: Job failed due to exception null
>>         at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:114)
>>         at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
>>         at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>         at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>         at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
>>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>         at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>         at java.lang.Thread.run(Thread.java:744)
>> 2015-02-23 13:11:49,673 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-118:ctx-adab6031 job-526) Complete async job-526,
>> jobStatus: FAILED, resultCode: 530, result:
>> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":
>> [],"errorcode":530,"errortext":"Job failed due to exception null"}
>> 2015-02-23 13:11:49,730 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>> (API-Job-Executor-118:ctx-adab6031 job-526) Done executing
>> org.apache.cloudstack.api.command.admin.vm.RemoveNicFromVMCmdByAdmin
>> for job-526
>> 2015-02-23 13:11:49,757 INFO  [o.a.c.f.j.i.AsyncJobMonitor]
>> (API-Job-Executor-118:ctx-adab6031 job-526) Remove job-526 from job
>> monitoring
>>
>>
>> /Sonali
>>
>> -----Original Message-----
>> From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
>> Sent: Monday, February 23, 2015 2:42 PM
>> To: users@cloudstack.apache.org
>> Subject: Re: unable to remove NIC from instance
>>
>> Sonali,
>>
>> this looks like a consequential error. You need to look further back into the log
to find the root cause. The start job is the a query for results, please find the start for
the actual api command.
>>
>> On Mon, Feb 23, 2015 at 6:55 AM, Sonali Jadhav <sonali@servercentralen.se>
wrote:
>>> I am trying to remove "Shared guest network". These are new logs,
>>>
>>> 2015-02-23 06:53:59,737 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-28b1e134)
Found 0 networks to update RvR status.
>>> 2015-02-23 06:54:01,726 DEBUG [c.c.a.ApiServlet]
>>> (http-6443-exec-3:ctx-6eb9497d) ===START===  115.114.134.114 -- GET
>>> command=queryAsyncJobResult&jobId=1fb40d3e-b0aa-4c68-b7d7-14bccb1b94c
>>> f
>>> &response=json&sessionkey=bjqAH0BCMrO1DvWk%2Beb9zCi1nuo%3D&_=14246707
>>> 3
>>> 8796
>>> 2015-02-23 06:54:01,818 DEBUG [c.c.a.ApiServlet]
>>> (http-6443-exec-3:ctx-6eb9497d ctx-007c94be) ===END===
>>> 115.114.134.114 -- GET
>>> command=queryAsyncJobResult&jobId=1fb40d3e-b0aa-4c68-b7d7-14bccb1b94c
>>> f
>>> &response=json&sessionkey=bjqAH0BCMrO1DvWk%2Beb9zCi1nuo%3D&_=14246707
>>> 3
>>> 8796
>>> 2015-02-23 06:54:01,993 ERROR [c.c.a.ApiAsyncJobDispatcher]
>>> (API-Job-Executor-105:ctx-b2370fa0 job-499) Unexpected exception
>>> while executing
>>> org.apache.cloudstack.api.command.admin.vm.RemoveNicFromVMCmdByAdmin
>>> java.lang.RuntimeException: Job failed due to exception null
>>>         at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:114)
>>>         at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
>>>         at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>>         at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>>         at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
>>>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>>         at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>         at java.lang.Thread.run(Thread.java:744)
>>> 2015-02-23 06:54:01,994 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (API-Job-Executor-105:ctx-b2370fa0 job-499) Complete async job-499,
>>> jobStatus: FAILED, resultCode: 530, result:
>>> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":
>>> [],"errorcode":530,"errortext":"Job failed due to exception null"}
>>> 2015-02-23 06:54:02,051 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>> (API-Job-Executor-105:ctx-b2370fa0 job-499) Done executing
>>> org.apache.cloudstack.api.command.admin.vm.RemoveNicFromVMCmdByAdmin
>>> for job-499
>>> 2015-02-23 06:54:02,077 INFO  [o.a.c.f.j.i.AsyncJobMonitor]
>>> (API-Job-Executor-105:ctx-b2370fa0 job-499) Remove job-499 from job
>>> monitoring
>>> 2015-02-23 06:54:03,942 DEBUG [c.c.s.StatsCollector] (StatsCollector-2:ctx-ce679066)
AutoScaling Monitor is running...
>>> 2015-02-23 06:54:04,727 DEBUG [c.c.a.ApiServlet]
>>> (http-6443-exec-4:ctx-e51e68c3) ===START===  115.114.134.114 -- GET
>>> command=queryAsyncJobResult&jobId=1fb40d3e-b0aa-4c68-b7d7-14bccb1b94c
>>> f
>>> &response=json&sessionkey=bjqAH0BCMrO1DvWk%2Beb9zCi1nuo%3D&_=14246707
>>> 4
>>> 1797
>>> 2015-02-23 06:54:04,802 DEBUG [c.c.a.ApiServlet]
>>> (http-6443-exec-4:ctx-e51e68c3 ctx-a70a6fff) ===END===
>>> 115.114.134.114 -- GET
>>> command=queryAsyncJobResult&jobId=1fb40d3e-b0aa-4c68-b7d7-14bccb1b94c
>>> f
>>> &response=json&sessionkey=bjqAH0BCMrO1DvWk%2Beb9zCi1nuo%3D&_=14246707
>>> 4
>>> 1797
>>>
>>>
>>> /Sonali
>>>
>>> -----Original Message-----
>>> From: Sonali Jadhav [mailto:sonali@servercentralen.se]
>>> Sent: Monday, February 23, 2015 10:25 AM
>>> To: users@cloudstack.apache.org
>>> Subject: RE: unable to remove NIC from instance
>>>
>>> Hi,
>>>
>>> Yes I have tried that, and issue is still same. Same logs.
>>>
>>> /Sonali
>>>
>>> -----Original Message-----
>>> From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
>>> Sent: Sunday, February 22, 2015 10:57 PM
>>> To: users@cloudstack.apache.org
>>> Subject: Re: unable to remove NIC from instance
>>>
>>> Sonil, Did you try what Gopalakrishnan suggested?
>>>
>>> Also can you expand on the version of cloudstack you are running?
>>>
>>> thanks,
>>>
>>> On Sun, Feb 22, 2015 at 12:24 PM, Sonali Jadhav <sonali@servercentralen.se>
wrote:
>>>> Anyone help?
>>>> /Sonali
>>>>
>>>>
>>>> ---- Sonali Jadhav wrote ----
>>>>
>>>> Anyone?
>>>>
>>>> Sent from my Sony Xperia™ smartphone
>>>>
>>>>
>>>> ---- Sonali Jadhav wrote ----
>>>>
>>>> Here are logs,
>>>>
>>>> 2015-02-20 14:26:34,122 DEBUG [c.c.a.m.AgentManagerImpl]
>>>> (AgentManager-Handler-12:null) SeqA 2-260495: Sending Seq 2-260495:
>>>> {
>>>> Ans: , MgmtId: 59778234354585, via: 2, Ver: v1, Flags: 100010,
>>>> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}
>>>> ]
>>>> }
>>>> 2015-02-20 14:26:34,638 DEBUG [o.a.c.f.j.d.VmWorkJobDaoImpl]
>>>> (Vm-Operations-Cleanup-1:ctx-53171ec5) Expunge completed work
>>>> job-483
>>>> 2015-02-20 14:26:34,640 DEBUG [c.c.u.d.T.Transaction]
>>>> (Vm-Operations-Cleanup-1:ctx-53171ec5) Rolling back the transaction:
>>>> Time = 2 Name =  Vm-Operations-Cleanup-1; called by
>>>> -TransactionLegacy.rollback:900-TransactionLegacy.removeUpTo:843-Tra
>>>> n
>>>> s
>>>> actionLegacy.close:667-TransactionContextInterceptor.invoke:36-Refle
>>>> c
>>>> t
>>>> iveMethodInvocation.proceed:161-ExposeInvocationInterceptor.invoke:9
>>>> 1
>>>> -
>>>> ReflectiveMethodInvocation.proceed:172-JdkDynamicAopProxy.invoke:204
>>>> -
>>>> $
>>>> Proxy182.expungeCompletedWorkJobs:-1-VirtualMachineManagerImpl$Clean
>>>> u
>>>> p
>>>> Task.runInContext:2389-ManagedContextRunnable$1.run:49-DefaultManage
>>>> d
>>>> C
>>>> ontext$1.call:56
>>>> 2015-02-20 14:26:34,658 ERROR [c.c.v.VirtualMachineManagerImpl]
>>>> (Vm-Operations-Cleanup-1:ctx-53171ec5) VM Operations failed due to
>>>> com.cloud.utils.exception.CloudRuntimeException: DB Exception on: com.mysql.jdbc.PreparedStatement@e7a49c5:
DELETE FROM async_job WHERE async_job.id= 483
>>>>         at com.cloud.utils.db.GenericDaoBase.expunge(GenericDaoBase.java:1178)
>>>>         at org.apache.cloudstack.framework.jobs.dao.VmWorkJobDaoImpl.expungeCompletedWorkJobs(VmWorkJobDaoImpl.java:149)
>>>>         at sun.reflect.GeneratedMethodAccessor183.invoke(Unknown Source)
>>>>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>         at java.lang.reflect.Method.invoke(Method.java:606)
>>>>         at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
>>>>         at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
>>>>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
>>>>         at com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
>>>>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
>>>>         at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
>>>>         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
>>>>         at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
>>>>         at com.sun.proxy.$Proxy182.expungeCompletedWorkJobs(Unknown Source)
>>>>         at com.cloud.vm.VirtualMachineManagerImpl$CleanupTask.runInContext(VirtualMachineManagerImpl.java:2389)
>>>>         at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>>>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>>>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>>>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>>>         at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>>>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>>>         at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
>>>>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>>>>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>>>>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>>         at java.lang.Thread.run(Thread.java:744)
>>>> Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException:
Cannot delete or update a parent row: a foreign key constraint fails (`cloud`.`async_job_join_map`,
CONSTRAINT `fk_async_job_join_map__join_job_id` FOREIGN KEY (`join_job_id`) REFERENCES `async_job`
(`id`))
>>>>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
>>>>         at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>>>>         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>>>         at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>>>>         at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
>>>>         at com.mysql.jdbc.Util.getInstance(Util.java:386)
>>>>         at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1040)
>>>>         at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3597)
>>>>         at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3529)
>>>>         at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1990)
>>>>         at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2151)
>>>>         at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2625)
>>>>         at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2119)
>>>>         at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2415)
>>>>         at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2333)
>>>>         at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2318)
>>>>         at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)
>>>>         at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)
>>>>         at com.cloud.utils.db.GenericDaoBase.expunge(GenericDaoBase.java:1169)
>>>>         ... 26 more
>>>> 2015-02-20 14:26:34,770 DEBUG [c.c.a.ApiServlet]
>>>> (http-6443-exec-4:ctx-46b96aa1) ===START===  192.168.34.1 -- GET
>>>> command=queryAsyncJobResult&jobId=ff043283-9481-46d2-bcd9-71fbdd0ab3
>>>> f
>>>> 1
>>>> &response=json&sessionkey=CkNBaXkKYXP6itmOoSTQVEKXiJo%3D&_=142443869
>>>> 6
>>>> 3
>>>> 84
>>>> 2015-02-20 14:26:34,797 DEBUG [c.c.u.AccountManagerImpl]
>>>> (http-6443-exec-4:ctx-46b96aa1 ctx-7b8c1717) Access to
>>>> Acct[35719f1c-832c-474d-8045-983ed17523a1-corp] granted to
>>>> Acct[35719f1c-832c-474d-8045-983ed17523a1-corp] by DomainChecker
>>>> 2015-02-20 14:26:34,858 DEBUG [c.c.a.ApiServlet]
>>>> (http-6443-exec-4:ctx-46b96aa1 ctx-7b8c1717) ===END===  192.168.34.1
>>>> -- GET
>>>> command=queryAsyncJobResult&jobId=ff043283-9481-46d2-bcd9-71fbdd0ab3
>>>> f
>>>> 1
>>>> &response=json&sessionkey=CkNBaXkKYXP6itmOoSTQVEKXiJo%3D&_=142443869
>>>> 6
>>>> 3
>>>> 84
>>>> 2015-02-20 14:26:35,024 ERROR [c.c.a.ApiAsyncJobDispatcher]
>>>> (API-Job-Executor-98:ctx-e2fe93d7 job-482) Unexpected exception
>>>> while executing
>>>> org.apache.cloudstack.api.command.user.vm.RemoveNicFromVMCmd
>>>> java.lang.RuntimeException: Job failed due to exception null
>>>>         at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:114)
>>>>         at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
>>>>         at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>>>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>>>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>>>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>>>         at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>>>         at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
>>>>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>>>         at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>>>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>>         at java.lang.Thread.run(Thread.java:744)
>>>> 2015-02-20 14:26:35,025 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>>> (API-Job-Executor-98:ctx-e2fe93d7 job-482) Complete async job-482,
>>>> jobStatus: FAILED, resultCode: 530, result:
>>>> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":
>>>> [],"errorcode":530,"errortext":"Job failed due to exception null"}
>>>> 2015-02-20 14:26:35,082 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
>>>> (API-Job-Executor-98:ctx-e2fe93d7 job-482) Done executing
>>>> org.apache.cloudstack.api.command.user.vm.RemoveNicFromVMCmd for
>>>> job-482
>>>> 2015-02-20 14:26:35,111 DEBUG [c.c.h.d.HostDaoImpl]
>>>> (ClusteredAgentManager Timer:ctx-c3486e1a) Resetting hosts suitable
>>>> for reconnect
>>>> 2015-02-20 14:26:35,113 INFO  [o.a.c.f.j.i.AsyncJobMonitor]
>>>> (API-Job-Executor-98:ctx-e2fe93d7 job-482) Remove job-482 from job
>>>> monitoring
>>>> 2015-02-20 14:26:35,114 DEBUG [c.c.h.d.HostDaoImpl]
>>>> (ClusteredAgentManager Timer:ctx-c3486e1a) Completed resetting hosts
>>>> suitable for reconnect
>>>> 2015-02-20 14:26:35,114 DEBUG [c.c.h.d.HostDaoImpl]
>>>> (ClusteredAgentManager Timer:ctx-c3486e1a) Acquiring hosts for
>>>> clusters already owned by this management server
>>>> 2015-02-20 14:26:35,116 DEBUG [c.c.h.d.HostDaoImpl]
>>>> (ClusteredAgentManager Timer:ctx-c3486e1a) Completed acquiring hosts
>>>> for clusters already owned by this management server
>>>> 2015-02-20 14:26:35,116 DEBUG [c.c.h.d.HostDaoImpl]
>>>> (ClusteredAgentManager Timer:ctx-c3486e1a) Acquiring hosts for
>>>> clusters not owned by any management server
>>>> 2015-02-20 14:26:35,117 DEBUG [c.c.h.d.HostDaoImpl]
>>>> (ClusteredAgentManager Timer:ctx-c3486e1a) Completed acquiring hosts
>>>> for clusters not owned by any management server
>>>> 2015-02-20 14:26:37,771 DEBUG [c.c.a.ApiServlet]
>>>> (http-6443-exec-2:ctx-1053809f) ===START===  192.168.34.1 -- GET
>>>> command=queryAsyncJobResult&jobId=ff043283-9481-46d2-bcd9-71fbdd0ab3
>>>> f
>>>> 1
>>>> &response=json&sessionkey=CkNBaXkKYXP6itmOoSTQVEKXiJo%3D&_=142443869
>>>> 9
>>>> 3
>>>> 85
>>>> 2015-02-20 14:26:37,798 DEBUG [c.c.u.AccountManagerImpl]
>>>> (http-6443-exec-2:ctx-1053809f ctx-5db06e08) Access to
>>>> Acct[35719f1c-832c-474d-8045-983ed17523a1-corp] granted to
>>>> Acct[35719f1c-832c-474d-8045-983ed17523a1-corp] by DomainChecker
>>>> 2015-02-20 14:26:37,857 DEBUG [c.c.a.ApiServlet]
>>>> (http-6443-exec-2:ctx-1053809f ctx-5db06e08) ===END===  192.168.34.1
>>>> -- GET
>>>> command=queryAsyncJobResult&jobId=ff043283-9481-46d2-bcd9-71fbdd0ab3
>>>> f
>>>> 1
>>>> &response=json&sessionkey=CkNBaXkKYXP6itmOoSTQVEKXiJo%3D&_=142443869
>>>> 9
>>>> 3
>>>> 85
>>>> 2015-02-20 14:26:37,975 DEBUG [c.c.a.m.AgentManagerImpl]
>>>> (AgentManager-Handler-2:null) Ping from 2
>>>> 2015-02-20 14:26:38,048 DEBUG [c.c.a.m.AgentManagerImpl]
>>>> (AgentManager-Handler-7:null) Ping from 9
>>>> 2015-02-20 14:26:40,509 WARN  [c.c.a.m.DirectAgentAttache]
>>>> (DirectAgentCronJob-261:ctx-f3e99173) Unable to get current status
>>>> on
>>>> 4(SeSolXS02)
>>>> 2015-02-20 14:26:40,687 WARN  [c.c.a.m.DirectAgentAttache]
>>>> (DirectAgentCronJob-130:ctx-8b85b559) Unable to get current status
>>>> on
>>>> 1(SeSolXS01)
>>>> 2015-02-20 14:26:40,747 WARN  [c.c.a.m.DirectAgentAttache]
>>>> (DirectAgentCronJob-131:ctx-e6c7b284) Unable to get current status
>>>> on
>>>> 5(SeSolXS03)
>>>> 2015-02-20 14:26:40,981 WARN  [c.c.a.m.DirectAgentAttache]
>>>> (DirectAgentCronJob-33:ctx-c6748244) Unable to get current status on
>>>> 6(SeSolXS04)
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> /Sonali
>>>>
>>>> -----Original Message-----
>>>> From: Gopalakrishnan S [mailto:gopal@assistanz.com]
>>>> Sent: Friday, February 20, 2015 7:12 PM
>>>> To: users@cloudstack.apache.org
>>>> Subject: Re: unable to remove NIC from instance
>>>>
>>>> Hi Sonali,
>>>>
>>>> It may be still routing that shared guest network. Try to restart your virutal
router and virtual machine or shutdown your VM before removing that NIC. let me know the logs
when you remove that NIC.
>>>>
>>>> Thank You.
>>>> Gopalakrishnan.S
>>>> Cloud Engineer - Fogpanel
>>>>
>>>>   ----- Original Message -----
>>>>   From: Sonali Jadhav
>>>>   To: users@cloudstack.apache.org
>>>>   Sent: Friday, February 20, 2015 7:06 PM
>>>>   Subject: unable to remove NIC from instance
>>>>
>>>>
>>>>   Hi,
>>>>
>>>>
>>>>
>>>>   I have one linux instance to which I had added "Shared guest network" NIC.
Then I added "isolated guest network" nic to it. Made "isolated guest network" NIC as default
and tried to remove "Shared guest network" NIC, but its failing with null exception. I have
attached logs.
>>>>
>>>>
>>>>
>>>>   Thanks,
>>>>
>>>>   /sonali
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Daan
>>>
>>>
>>>
>>>
>>
>>
>>
>> --
>> Daan
>>
>>
>
>
>
> --
> Daan
>
>



-- 
Daan

Mime
View raw message