cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abhinav Roy (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CLOUDSTACK-5676) Live migration of VM is failing with a NPE
Date Mon, 30 Dec 2013 12:48:53 GMT
Abhinav Roy created CLOUDSTACK-5676:
---------------------------------------

             Summary: Live migration of VM is failing with a NPE
                 Key: CLOUDSTACK-5676
                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5676
             Project: CloudStack
          Issue Type: Bug
      Security Level: Public (Anyone can view this level - this is the default.)
          Components: Management Server
    Affects Versions: 4.3.0
         Environment: hyperv , 4.3
            Reporter: Abhinav Roy
            Priority: Blocker
             Fix For: 4.3.0


Steps to Reproduce: 
================ 
1.Bring up CS in advanced zone with 2 hosts in a hyper-v cluster using CIFS for both primary
and secondary storage 
2.Deploy one or two guest vms using default cent os template. 
3.Migrate one of the VMs from Host1 to Host2

Expected Behaviour: 
============= 
Live VM migration should be successful

Observed Behaviour: 
=========== 
Live VM migration fails with :

013-12-30 16:41:29,564 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-1:ctx-2ba86ed3
ctx-ca0be041) Sync job-95 execution on object VmWorkJobQueue.16
2013-12-30 16:41:31,261 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-c85fb697)
Execute sync-queue item: SyncQueueItemVO {id:30, queueId: 27, contentType: AsyncJob, contentId:
95, lastProcessMsid: null, lastprocessNumber: null, lastProcessTime: null, created: Mon Dec
30 16:41:29 IST 2013}
2013-12-30 16:41:31,263 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-c85fb697)
Schedule queued job-95
2013-12-30 16:41:31,277 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (Job-Executor-2:ctx-ba975c56)
Add job-95 into job monitoring
2013-12-30 16:41:31,278 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-2:ctx-ba975c56)
Executing AsyncJobVO {id:95, userId: 2, accountId: 2, instanceType: null, instanceId: null,
cmd: com.cloud.vm.VmWorkMigrate, cmdInfo: rO0ABXNyABpjb20uY2xvdWQudm0uVm1Xb3JrTWlncmF0ZRdxQXtPtzYqAgAGSgAJc3JjSG9zdElkTAAJY2x1c3RlcklkdAAQTGphdmEvbGFuZy9Mb25nO0wABmhvc3RJZHEAfgABTAAFcG9kSWRxAH4AAUwAB3N0b3JhZ2V0AA9MamF2YS91dGlsL01hcDtMAAZ6b25lSWRxAH4AAXhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAEHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAXNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cgAQamF2YS5sYW5nLk51bWJlcoaslR0LlOCLAgAAeHAAAAAAAAAAAXNxAH4ABwAAAAAAAAAEcQB-AAlwcQB-AAk,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid:
280320865129348, completeMsid: null, lastUpdated: null, lastPolled: null, created: Mon Dec
30 16:41:29 IST 2013}
2013-12-30 16:41:31,279 DEBUG [c.c.v.VmWorkJobDispatcher] (Job-Executor-2:ctx-ba975c56) Run
VM work job: com.cloud.vm.VmWorkMigrate

2013-12-30 16:41:31,287 ERROR [c.c.v.VmWorkJobDispatcher] (Job-Executor-2:ctx-ba975c56 ctx-ca0be041)
Unable to complete AsyncJobVO {id:95, userId: 2, accountId: 2, instanceType: null, instanceId:
null, cmd: com.cloud.vm.VmWorkMigrate, cmdInfo: rO0ABXNyABpjb20uY2xvdWQudm0uVm1Xb3JrTWlncmF0ZRdxQXtPtzYqAgAGSgAJc3JjSG9zdElkTAAJY2x1c3RlcklkdAAQTGphdmEvbGFuZy9Mb25nO0wABmhvc3RJZHEAfgABTAAFcG9kSWRxAH4AAUwAB3N0b3JhZ2V0AA9MamF2YS91dGlsL01hcDtMAAZ6b25lSWRxAH4AAXhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAEHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAXNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cgAQamF2YS5sYW5nLk51bWJlcoaslR0LlOCLAgAAeHAAAAAAAAAAAXNxAH4ABwAAAAAAAAAEcQB-AAlwcQB-AAk,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid:
280320865129348, completeMsid: null, lastUpdated: null, lastPolled: null, created: Mon Dec
30 16:41:29 IST 2013}
java.lang.NullPointerException
        at com.cloud.vm.VmWorkMigrate.getDeployDestination(VmWorkMigrate.java:60)
        at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4743)
        at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:99)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:522)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
2013-12-30 16:41:31,291 ERROR [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-2:ctx-ba975c56)
Unexpected exception
java.lang.NullPointerException
        at com.google.gson.JsonPrimitive.isPrimitiveOrString(JsonPrimitive.java:316)
        at com.google.gson.JsonPrimitive.setValue(JsonPrimitive.java:97)
        at com.google.gson.JsonPrimitive.<init>(JsonPrimitive.java:67)
        at org.apache.cloudstack.framework.jobs.impl.JobSerializerHelper$ThrowableTypeAdapter.serialize(JobSerializerHelper.java:195)
        at org.apache.cloudstack.framework.jobs.impl.JobSerializerHelper$ThrowableTypeAdapter.serialize(JobSerializerHelper.java:158)
        at com.google.gson.JsonSerializationVisitor.findAndInvokeCustomSerializer(JsonSerializationVisitor.java:184)
        at com.google.gson.JsonSerializationVisitor.visitFieldUsingCustomHandler(JsonSerializationVisitor.java:204)
        at com.google.gson.ReflectingFieldNavigator.visitFieldsReflectively(ReflectingFieldNavigator.java:63)
        at com.google.gson.ObjectNavigator.accept(ObjectNavigator.java:120)
        at com.google.gson.JsonSerializationContextDefault.serialize(JsonSerializationContextDefault.java:62)
        at com.google.gson.JsonSerializationContextDefault.serialize(JsonSerializationContextDefault.java:53)
        at com.google.gson.Gson.toJsonTree(Gson.java:220)
        at com.google.gson.Gson.toJson(Gson.java:260)
        at com.google.gson.Gson.toJson(Gson.java:240)
        at org.apache.cloudstack.framework.jobs.impl.JobSerializerHelper.toSerializedString(JobSerializerHelper.java:65)
        at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:105)
        at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:522)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
2013-12-30 16:41:31,296 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-2:ctx-ba975c56)
Complete async job-95, jobStatus: FAILED, resultCode: 530, result: null
2013-12-30 16:41:31,313 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (Job-Executor-3:ctx-71346dc1)
Add job-94 into job monitoring
2013-12-30 16:41:31,313 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-3:ctx-71346dc1)
Executing AsyncJobVO {id:94, userId: 2, accountId: 2, instanceType: None, instanceId: null,
cmd: org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd, cmdInfo: {"response":"json","sessionkey":"lIed4xLaMsDL4yCyAJYyUK0pAJk\u003d","virtualmachineid":"462e6bfc-7c79-4290-9967-8b897ada09e1","cmdEventType":"VM.MIGRATE","hostid":"655f6fb2-82b7-4388-9462-3c15f824e5f9","ctxUserId":"2","httpmethod":"GET","_":"1388401853884","ctxAccountId":"2","ctxStartEventId":"216"},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid:
280320865129348, completeMsid: null, lastUpdated: null, lastPolled: null, created: Mon Dec
30 16:41:29 IST 2013}
2013-12-30 16:41:31,328 WARN  [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-3:ctx-71346dc1)
job-94 is scheduled for wakeup run, but there is no joining info anymore
2013-12-30 16:41:31,333 ERROR [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-3:ctx-71346dc1)
Unable to find a wakeup dispatcher from the joined job: AsyncJobVO {id:94, userId: 2, accountId:
2, instanceType: None, instanceId: null, cmd: org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd,
cmdInfo: {"response":"json","sessionkey":"lIed4xLaMsDL4yCyAJYyUK0pAJk\u003d","virtualmachineid":"462e6bfc-7c79-4290-9967-8b897ada09e1","cmdEventType":"VM.MIGRATE","hostid":"655f6fb2-82b7-4388-9462-3c15f824e5f9","ctxUserId":"2","httpmethod":"GET","_":"1388401853884","ctxAccountId":"2","ctxStartEventId":"216"},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid:
280320865129348, completeMsid: null, lastUpdated: null, lastPolled: null, created: Mon Dec
30 16:41:29 IST 2013}
2013-12-30 16:41:31,334 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-3:ctx-71346dc1)
Done executing org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd for job-94
2013-12-30 16:41:31,336 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (Job-Executor-2:ctx-ba975c56)
Sync queue (27) is currently empty
2013-12-30 16:41:31,337 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (Job-Executor-2:ctx-ba975c56)
Remove job-95 from job monitoring
2013-12-30 16:41:31,347 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (Job-Executor-3:ctx-71346dc1)
Remove job-94 from job monitoring
2013-12-30 16:41:31,364 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-1:ctx-2ba86ed3
ctx-ca0be041) Complete async job-94, jobStatus: SUCCEEDED, resultCode: 0, result: org.apache.cloudstack.api.response.UserVmResponse/virtualmachine/{"id":"462e6bfc-7c79-4290-9967-8b897ada09e1","name":"av8","displayname":"av8","account":"admin","domainid":"3478c004-7129-11e3-89a3-fef34996d384","domain":"ROOT","created":"2013-12-30T16:31:47+0530","state":"Running","haenable":false,"zoneid":"aaa51135-ad9e-4053-9023-17b7923b8694","zonename":"BLR","hostid":"cd56ae63-24c8-4e55-be45-c133f9ecb50f","hostname":"10.102.192.14","templateid":"3483f6ae-7129-11e3-89a3-fef34996d384","templatename":"CentOS
6.4(64-bit) GUI (Hyperv)","templatedisplaytext":"CentOS 6.4 (64-bit) GUI (Hyperv)","passwordenabled":false,"serviceofferingid":"869e9873-ab4c-48c1-a57b-58b73f2939ec","serviceofferingname":"Small
Instance","cpunumber":1,"cpuspeed":500,"memory":512,"cpuused":"0%","networkkbsread":1,"networkkbswrite":1,"guestosid":"34a20e96-7129-11e3-89a3-fef34996d384","rootdeviceid":0,"rootdevicetype":"ROOT","securitygroup":[],"nic":[{"id":"e897a7eb-4bea-4335-8925-a240c95df9da","networkid":"80e01c55-75b3-4e7e-9668-fe46ecae4ea8","networkname":"an1","netmask":"255.255.255.0","gateway":"10.1.1.1","ipaddress":"10.1.1.7","isolationuri":"vlan://680","broadcasturi":"vlan://680","traffictype":"Guest","type":"Isolated","isdefault":true,"macaddress":"02:00:08:a2:00:09"}],"hypervisor":"Hyperv","instancename":"i-2-16-VM","tags":[],"affinitygroup":[],"displayvm":true,"isdynamicallyscalable":false}
2013-12-30 16:41:31,372 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-1:ctx-2ba86ed3)
Done executing org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd for job-94
2013-12-30 16:41:31,382 INFO  [o.a.c.f.j.i.AsyncJobMonitor] (Job-Executor-1:ctx-2ba86ed3)
Remove job-94 from job monitoring
2013-12-30 16:41:32,507 DEBUG [c.c.a.ApiServlet] (catalina-exec-16:ctx-5f391aea) ===START===
 10.144.7.20 -- GET  command=queryAsyncJobResult&jobId=f94905d3-a6e7-4119-aa2a-a58c673979c4&response=json&sessionkey=lIed4xLaMsDL4yCyAJYyUK0pAJk%3D&_=1388401856795



Workaround :
=========================
Add a new entry in cloud.configurations table with 

vm.job.enabled set to "false" 

restart management server. After this Live VM migration works fine.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message