cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ram Ganesh (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CLOUDSTACK-5676) Live migration of VM is failing with a NPE
Date Wed, 01 Jan 2014 06:10:51 GMT

     [ https://issues.apache.org/jira/browse/CLOUDSTACK-5676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ram Ganesh updated CLOUDSTACK-5676:
-----------------------------------

    Labels: vmsync  (was: )

> Live migration of VM is failing with a NPE
> ------------------------------------------
>
>                 Key: CLOUDSTACK-5676
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5676
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Management Server
>    Affects Versions: 4.3.0
>         Environment: hyperv , 4.3
>            Reporter: Abhinav Roy
>            Assignee: Devdeep Singh
>            Priority: Blocker
>              Labels: vmsync
>             Fix For: 4.3.0
>
>
> Steps to Reproduce: 
> ================ 
> 1.Bring up CS in advanced zone with 2 hosts in a hyper-v cluster using CIFS for both
primary and secondary storage 
> 2.Deploy one or two guest vms using default cent os template. 
> 3.Migrate one of the VMs from Host1 to Host2
> Expected Behaviour: 
> ============= 
> Live VM migration should be successful
> Observed Behaviour: 
> =========== 
> Live VM migration fails with :
> 013-12-30 16:41:29,564 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-1:ctx-2ba86ed3
ctx-ca0be041) Sync job-95 execution on object VmWorkJobQueue.16
> 2013-12-30 16:41:31,261 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-c85fb697)
Execute sync-queue item: SyncQueueItemVO {id:30, queueId: 27, contentType: AsyncJob, contentId:
95, lastProcessMsid: null, lastprocessNumber: null, lastProcessTime: null, created: Mon Dec
30 16:41:29 IST 2013}
> 2013-12-30 16:41:31,263 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-c85fb697)
Schedule queued job-95
> 2013-12-30 16:41:31,277 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (Job-Executor-2:ctx-ba975c56)
Add job-95 into job monitoring
> 2013-12-30 16:41:31,278 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-2:ctx-ba975c56)
Executing AsyncJobVO {id:95, userId: 2, accountId: 2, instanceType: null, instanceId: null,
cmd: com.cloud.vm.VmWorkMigrate, cmdInfo: rO0ABXNyABpjb20uY2xvdWQudm0uVm1Xb3JrTWlncmF0ZRdxQXtPtzYqAgAGSgAJc3JjSG9zdElkTAAJY2x1c3RlcklkdAAQTGphdmEvbGFuZy9Mb25nO0wABmhvc3RJZHEAfgABTAAFcG9kSWRxAH4AAUwAB3N0b3JhZ2V0AA9MamF2YS91dGlsL01hcDtMAAZ6b25lSWRxAH4AAXhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAEHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAXNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cgAQamF2YS5sYW5nLk51bWJlcoaslR0LlOCLAgAAeHAAAAAAAAAAAXNxAH4ABwAAAAAAAAAEcQB-AAlwcQB-AAk,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid:
280320865129348, completeMsid: null, lastUpdated: null, lastPolled: null, created: Mon Dec
30 16:41:29 IST 2013}
> 2013-12-30 16:41:31,279 DEBUG [c.c.v.VmWorkJobDispatcher] (Job-Executor-2:ctx-ba975c56)
Run VM work job: com.cloud.vm.VmWorkMigrate
> 2013-12-30 16:41:31,287 ERROR [c.c.v.VmWorkJobDispatcher] (Job-Executor-2:ctx-ba975c56
ctx-ca0be041) Unable to complete AsyncJobVO {id:95, userId: 2, accountId: 2, instanceType:
null, instanceId: null, cmd: com.cloud.vm.VmWorkMigrate, cmdInfo: rO0ABXNyABpjb20uY2xvdWQudm0uVm1Xb3JrTWlncmF0ZRdxQXtPtzYqAgAGSgAJc3JjSG9zdElkTAAJY2x1c3RlcklkdAAQTGphdmEvbGFuZy9Mb25nO0wABmhvc3RJZHEAfgABTAAFcG9kSWRxAH4AAUwAB3N0b3JhZ2V0AA9MamF2YS91dGlsL01hcDtMAAZ6b25lSWRxAH4AAXhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAEHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAXNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cgAQamF2YS5sYW5nLk51bWJlcoaslR0LlOCLAgAAeHAAAAAAAAAAAXNxAH4ABwAAAAAAAAAEcQB-AAlwcQB-AAk,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid:
280320865129348, completeMsid: null, lastUpdated: null, lastPolled: null, created: Mon Dec
30 16:41:29 IST 2013}
> java.lang.NullPointerException
>         at com.cloud.vm.VmWorkMigrate.getDeployDestination(VmWorkMigrate.java:60)
>         at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4743)
>         at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:99)
>         at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:522)
>         at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>         at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2013-12-30 16:41:31,291 ERROR [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-2:ctx-ba975c56)
Unexpected exception
> java.lang.NullPointerException
>         at com.google.gson.JsonPrimitive.isPrimitiveOrString(JsonPrimitive.java:316)
>         at com.google.gson.JsonPrimitive.setValue(JsonPrimitive.java:97)
>         at com.google.gson.JsonPrimitive.<init>(JsonPrimitive.java:67)
>         at org.apache.cloudstack.framework.jobs.impl.JobSerializerHelper$ThrowableTypeAdapter.serialize(JobSerializerHelper.java:195)
>         at org.apache.cloudstack.framework.jobs.impl.JobSerializerHelper$ThrowableTypeAdapter.serialize(JobSerializerHelper.java:158)
>         at com.google.gson.JsonSerializationVisitor.findAndInvokeCustomSerializer(JsonSerializationVisitor.java:184)
>         at com.google.gson.JsonSerializationVisitor.visitFieldUsingCustomHandler(JsonSerializationVisitor.java:204)
>         at com.google.gson.ReflectingFieldNavigator.visitFieldsReflectively(ReflectingFieldNavigator.java:63)
>         at com.google.gson.ObjectNavigator.accept(ObjectNavigator.java:120)
>         at com.google.gson.JsonSerializationContextDefault.serialize(JsonSerializationContextDefault.java:62)
>         at com.google.gson.JsonSerializationContextDefault.serialize(JsonSerializationContextDefault.java:53)
>         at com.google.gson.Gson.toJsonTree(Gson.java:220)
>         at com.google.gson.Gson.toJson(Gson.java:260)
>         at com.google.gson.Gson.toJson(Gson.java:240)
>         at org.apache.cloudstack.framework.jobs.impl.JobSerializerHelper.toSerializedString(JobSerializerHelper.java:65)
>         at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:105)
>         at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:522)
>         at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>         at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>         at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2013-12-30 16:41:31,296 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-2:ctx-ba975c56)
Complete async job-95, jobStatus: FAILED, resultCode: 530, result: null
> 2013-12-30 16:41:31,313 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (Job-Executor-3:ctx-71346dc1)
Add job-94 into job monitoring
> 2013-12-30 16:41:31,313 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-3:ctx-71346dc1)
Executing AsyncJobVO {id:94, userId: 2, accountId: 2, instanceType: None, instanceId: null,
cmd: org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd, cmdInfo: {"response":"json","sessionkey":"lIed4xLaMsDL4yCyAJYyUK0pAJk\u003d","virtualmachineid":"462e6bfc-7c79-4290-9967-8b897ada09e1","cmdEventType":"VM.MIGRATE","hostid":"655f6fb2-82b7-4388-9462-3c15f824e5f9","ctxUserId":"2","httpmethod":"GET","_":"1388401853884","ctxAccountId":"2","ctxStartEventId":"216"},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid:
280320865129348, completeMsid: null, lastUpdated: null, lastPolled: null, created: Mon Dec
30 16:41:29 IST 2013}
> 2013-12-30 16:41:31,328 WARN  [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-3:ctx-71346dc1)
job-94 is scheduled for wakeup run, but there is no joining info anymore
> 2013-12-30 16:41:31,333 ERROR [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-3:ctx-71346dc1)
Unable to find a wakeup dispatcher from the joined job: AsyncJobVO {id:94, userId: 2, accountId:
2, instanceType: None, instanceId: null, cmd: org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd,
cmdInfo: {"response":"json","sessionkey":"lIed4xLaMsDL4yCyAJYyUK0pAJk\u003d","virtualmachineid":"462e6bfc-7c79-4290-9967-8b897ada09e1","cmdEventType":"VM.MIGRATE","hostid":"655f6fb2-82b7-4388-9462-3c15f824e5f9","ctxUserId":"2","httpmethod":"GET","_":"1388401853884","ctxAccountId":"2","ctxStartEventId":"216"},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid:
280320865129348, completeMsid: null, lastUpdated: null, lastPolled: null, created: Mon Dec
30 16:41:29 IST 2013}
> 2013-12-30 16:41:31,334 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-3:ctx-71346dc1)
Done executing org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd for job-94
> 2013-12-30 16:41:31,336 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (Job-Executor-2:ctx-ba975c56)
Sync queue (27) is currently empty
> 2013-12-30 16:41:31,337 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (Job-Executor-2:ctx-ba975c56)
Remove job-95 from job monitoring
> 2013-12-30 16:41:31,347 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (Job-Executor-3:ctx-71346dc1)
Remove job-94 from job monitoring
> 2013-12-30 16:41:31,364 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-1:ctx-2ba86ed3
ctx-ca0be041) Complete async job-94, jobStatus: SUCCEEDED, resultCode: 0, result: org.apache.cloudstack.api.response.UserVmResponse/virtualmachine/{"id":"462e6bfc-7c79-4290-9967-8b897ada09e1","name":"av8","displayname":"av8","account":"admin","domainid":"3478c004-7129-11e3-89a3-fef34996d384","domain":"ROOT","created":"2013-12-30T16:31:47+0530","state":"Running","haenable":false,"zoneid":"aaa51135-ad9e-4053-9023-17b7923b8694","zonename":"BLR","hostid":"cd56ae63-24c8-4e55-be45-c133f9ecb50f","hostname":"10.102.192.14","templateid":"3483f6ae-7129-11e3-89a3-fef34996d384","templatename":"CentOS
6.4(64-bit) GUI (Hyperv)","templatedisplaytext":"CentOS 6.4 (64-bit) GUI (Hyperv)","passwordenabled":false,"serviceofferingid":"869e9873-ab4c-48c1-a57b-58b73f2939ec","serviceofferingname":"Small
Instance","cpunumber":1,"cpuspeed":500,"memory":512,"cpuused":"0%","networkkbsread":1,"networkkbswrite":1,"guestosid":"34a20e96-7129-11e3-89a3-fef34996d384","rootdeviceid":0,"rootdevicetype":"ROOT","securitygroup":[],"nic":[{"id":"e897a7eb-4bea-4335-8925-a240c95df9da","networkid":"80e01c55-75b3-4e7e-9668-fe46ecae4ea8","networkname":"an1","netmask":"255.255.255.0","gateway":"10.1.1.1","ipaddress":"10.1.1.7","isolationuri":"vlan://680","broadcasturi":"vlan://680","traffictype":"Guest","type":"Isolated","isdefault":true,"macaddress":"02:00:08:a2:00:09"}],"hypervisor":"Hyperv","instancename":"i-2-16-VM","tags":[],"affinitygroup":[],"displayvm":true,"isdynamicallyscalable":false}
> 2013-12-30 16:41:31,372 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-1:ctx-2ba86ed3)
Done executing org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd for job-94
> 2013-12-30 16:41:31,382 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (Job-Executor-1:ctx-2ba86ed3)
Remove job-94 from job monitoring
> 2013-12-30 16:41:32,507 DEBUG [c.c.a.ApiServlet] (catalina-exec-16:ctx-5f391aea) ===START===
 10.144.7.20 -- GET  command=queryAsyncJobResult&jobId=f94905d3-a6e7-4119-aa2a-a58c673979c4&response=json&sessionkey=lIed4xLaMsDL4yCyAJYyUK0pAJk%3D&_=1388401856795
> Workaround :
> =========================
> Add a new entry in cloud.configurations table with 
> vm.job.enabled set to "false" 
> restart management server. After this Live VM migration works fine.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message