cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "angeline shen (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CLOUDSTACK-4125) [Upgrade] after upgrade 3.0.7 GA -> 4.2 deploy guest VM fail
Date Wed, 07 Aug 2013 01:20:48 GMT
angeline shen created CLOUDSTACK-4125:
-----------------------------------------

             Summary: [Upgrade] after upgrade 3.0.7 GA -> 4.2 deploy guest VM fail
                 Key: CLOUDSTACK-4125
                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4125
             Project: CloudStack
          Issue Type: Bug
      Security Level: Public (Anyone can view this level - this is the default.)
          Components: Management Server
    Affects Versions: 4.2.0
         Environment: MS      10.223.195.62       campo 4.2
host     10.223.51.3          XS 6.1
            Reporter: angeline shen
            Priority: Blocker
             Fix For: 4.2.0


MS 10.223.195.62 build 3.0.7 GA upgrade 4.2 build host 10.223.51.3 XS 6.1

1. MS install 3.0.7 GA,
2. create advanced zone with SG
3. create several networks with different subnets and same vlan
                       vlan GW start IP end IP
----------------------------------------------------------------------------------------------------------
guest network 1250 10.223.125.1 10.223.125.13 10.223.125.18
 
1250-66to68 1250 10.223.125.65 10.223.125.66 10.223.125.68
1250-130to132 1250 10.223.125.129 10.223.125.130 10.223.125.132

4. start VMs on each of these networks
125.18 125.16 125.14 125.68 125.67 125.131 125.132
no ingress or egress rules :
VMs can ssh + ping other VMs in all subnets
VMs can egress internet

add ingress TCP icmp egress TCP egress

5. make sure VM work well
6. upgrade to 4.2
7. existing guest VM works welll
8. check multiple subnets
     - add new IP range to subnet in network
     - start new VM in network in second subnet  -  FAILed

2013-08-06 17:00:35,556 DEBUG [storage.volume.VolumeServiceImpl] (Job-Executor-8:job-21 =
[ f2595277-039b-4166-a8c3-01044c1b4cec ]) Acquire lock on VMTemplateStoragePool 2 with timeout
3600 seconds
2013-08-06 17:00:35,557 INFO  [storage.volume.VolumeServiceImpl] (Job-Executor-8:job-21 =
[ f2595277-039b-4166-a8c3-01044c1b4cec ]) lock is acquired for VMTemplateStoragePool 2
2013-08-06 17:00:35,559 DEBUG [storage.volume.VolumeServiceImpl] (Job-Executor-8:job-21 =
[ f2595277-039b-4166-a8c3-01044c1b4cec ]) failed to create template on storage
com.cloud.utils.exception.CloudRuntimeException: Failed to update state
        at org.apache.cloudstack.storage.image.store.TemplateObject.processEvent(TemplateObject.java:164)
        at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createBaseImageAsync(VolumeServiceImpl.java:433)
        at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:538)
        at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2488)
        at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2545)
        at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:934)
        at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:624)
        at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:227)
        at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3408)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2968)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2954)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
        at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
Caused by: com.cloud.utils.fsm.NoTransitionException: Unable to transition to a new state
from null via CreateOnlyRequested
        at com.cloud.utils.fsm.StateMachine2.getNextState(StateMachine2.java:83)
        at com.cloud.utils.fsm.StateMachine2.transitTo(StateMachine2.java:100)
        at org.apache.cloudstack.storage.datastore.ObjectInDataStoreManagerImpl.update(ObjectInDataStoreManagerImpl.java:258)
        at org.apache.cloudstack.storage.image.store.TemplateObject.processEvent(TemplateObject.java:162)
        ... 25 more
2013-08-06 17:00:35,579 INFO  [storage.volume.VolumeServiceImpl] (Job-Executor-8:job-21 =
[ f2595277-039b-4166-a8c3-01044c1b4cec ]) releasing lock for VMTemplateStoragePool 2
2013-08-06 17:00:35,579 ERROR [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-8:job-21
= [ f2595277-039b-4166-a8c3-01044c1b4cec ]) Failed to start instance VM[User|z1upgrade1250-130to132]
com.cloud.utils.exception.CloudRuntimeException: Failed to update state
        at org.apache.cloudstack.storage.image.store.TemplateObject.processEvent(TemplateObject.java:164)
        at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createBaseImageAsync(VolumeServiceImpl.java:437)
        at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:538)
        at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2488)
        at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2545)
        at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:934)
        at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:624)
        at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:227)
        at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3408)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2968)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2954)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
        at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
Caused by: com.cloud.utils.fsm.NoTransitionException: Unable to transition to a new state
from null via OperationFailed
        at com.cloud.utils.fsm.StateMachine2.getNextState(StateMachine2.java:83)
        at com.cloud.utils.fsm.StateMachine2.transitTo(StateMachine2.java:100)
        at org.apache.cloudstack.storage.datastore.ObjectInDataStoreManagerImpl.update(ObjectInDataStoreManagerImpl.java:258)
        at org.apache.cloudstack.storage.image.store.TemplateObject.processEvent(TemplateObject.java:162)
2013-08-06 17:00:35,584 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-8:job-21
= [ f2595277-039b-4166-a8c3-01044c1b4cec ]) Cleaning up resources for the vm VM[User|z1upgrade1250-130to132]
in Starting state
2013-08-06 17:00:35,585 DEBUG [agent.transport.Request] (Job-Executor-8:job-21 = [ f2595277-039b-4166-a8c3-01044c1b4cec
]) Seq 1-1201344676: Sending  { Cmd , MgmtId: 7264031213276, via: 1, Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"i-2-12-VM","wait":0}}]
}
2013-08-06 17:00:35,585 DEBUG [agent.transport.Request] (Job-Executor-8:job-21 = [ f2595277-039b-4166-a8c3-01044c1b4cec
]) Seq 1-120

2013-08-06 17:00:35,852 DEBUG [cloud.vm.UserVmManagerImpl] (Job-Executor-8:job-21 = [ f2595277-039b-4166-a8c3-01044c1b4cec
]) Destroying vm VM[User|z1upgrade1250-130to132] as it failed to create on Host with Id:null
2013-08-06 17:00:35,866 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-8:job-21
= [ f2595277-039b-4166-a8c3-01044c1b4cec ]) VM state transitted from :Stopped to Error with
event: OperationFailedToErrorvm's original host id: null new host id: null host id before
state transition: null
2013-08-06 17:00:36,131 DEBUG [cloud.api.ApiServlet] (catalina-exec-20:null) ===START=== 
10.216.133.70 -- GET  command=queryAsyncJobResult&jobId=f2595277-039b-4166-a8c3-01044c1b4cec&response=json&sessionkey=C2NtUAEudxOaSrj3%2B%2ByPxrOyXMo%3D&_=1375834266650
2013-08-06 17:00:36,158 DEBUG [cloud.api.ApiServlet] (catalina-exec-20:null) ===END===  10.216.133.70
-- GET  command=queryAsyncJobResult&jobId=f2595277-039b-4166-a8c3-01044c1b4cec&response=json&sessionkey=C2NtUAEudxOaSrj3%2B%2ByPxrOyXMo%3D&_=1375834266650
2013-08-06 17:00:36,176 WARN  [apache.cloudstack.alerts] (Job-Executor-8:job-21 = [ f2595277-039b-4166-a8c3-01044c1b4cec
])  alertType:: 8 // dataCenterId:: 1 // podId:: 1 // clusterId:: null // message:: Failed
to deploy Vm with Id: 12, on Host with Id: null
2013-08-06 17:00:36,209 INFO  [user.vm.DeployVMCmd] (Job-Executor-8:job-21 = [ f2595277-039b-4166-a8c3-01044c1b4cec
]) com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment
for VM[User|z1upgrade1250-130to132]Scope=interface com.cloud.dc.DataCenter; id=1
2013-08-06 17:00:36,209 INFO  [user.vm.DeployVMCmd] (Job-Executor-8:job-21 = [ f2595277-039b-4166-a8c3-01044c1b4cec
]) Unable to create a deployment for VM[User|z1upgrade1250-130to132]
com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for
VM[User|z1upgrade1250-130to132]Scope=interface com.cloud.dc.DataCenter; id=1
        at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:888)
        at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:624)
        at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:237)
        at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3408)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2968)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2954)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
        at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
2013-08-06 17:00:36,210 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-8:job-21 = [
f2595277-039b-4166-a8c3-01044c1b4cec ]) Complete async job-21 = [ f2595277-039b-4166-a8c3-01044c1b4cec
], jobStatus: 2, resultCode: 530, result: Error Code: 533 Error text: Unable to create a deployment
for VM[User|z1upgrade1250-130to132]
2013-08-06 17:00:39,128 DEBUG [cloud.api.ApiServlet] (catalina-exec-23:null) ===START=== 
10.216.133.70 -- GET  command=queryAsyncJobResult&jobId=f2595277-039b-4166-a8c3-01044c1b4cec&response=json&sessionkey=C2NtUAEudxOaSrj3%2B%2ByPxrOyXMo%3D&_=1375834269647
2013-08-06 17:00:39,141 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-23:null) Async
job-21 = [ f2595277-039b-4166-a8c3-01044c1b4cec ] completed


------------------------------------------------------------------------------------------



9. destroy 1 guest VM.    Deploy guest VM :  VM  remain in STOPPED state

2013-08-06 17:24:57,230 DEBUG [cloud.deploy.DeploymentPlanningManagerImpl] (Job-Executor-1:job-24
= [ 85b0ca6d-7586-4225-a9aa-b772ea99e3a8 ]) No suitable hosts found
2013-08-06 17:24:57,230 DEBUG [cloud.deploy.DeploymentPlanningManagerImpl] (Job-Executor-1:job-24
= [ 85b0ca6d-7586-4225-a9aa-b772ea99e3a8 ]) No suitable hosts found under this Cluster: 1
2013-08-06 17:24:57,232 DEBUG [cloud.deploy.DeploymentPlanningManagerImpl] (Job-Executor-1:job-24
= [ 85b0ca6d-7586-4225-a9aa-b772ea99e3a8 ]) Could not find suitable Deployment Destinatio
n for this VM under any clusters, returning. 
2013-08-06 17:24:57,232 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-1:job-24 = [ 85b0ca6d-7586-4225-a9aa-b772ea99e3a8
]) Searching resources only under specified Pod: 1
2013-08-06 17:24:57,232 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-1:job-24 = [ 85b0ca6d-7586-4225-a9aa-b772ea99e3a8
]) Listing clusters in order of aggregate capacity, that have (atleast one host with) enough
CPU and RAM capacity under this Pod: 1
2013-08-06 17:24:57,235 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-1:job-24 = [ 85b0ca6d-7586-4225-a9aa-b772ea99e3a8
]) Removing from the clusterId list these clusters from avoid set: [1]
2013-08-06 17:24:57,235 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-1:job-24 = [ 85b0ca6d-7586-4225-a9aa-b772ea99e3a8
]) No clusters found after removing disabled clusters and clusters in avoid list, returning.
2013-08-06 17:24:57,280 DEBUG [cloud.api.ApiServlet] (catalina-exec-14:null) ===START=== 
10.216.133.64 -- GET  command=queryAsyncJobResult&jobId=85b0ca6d-7586-4225-a9aa-b772ea99e3a8&response=json&sessionkey=kxfo3DkYEzBv44N8ynxFwuMn6bE%3D&_=1375835530701
2013-08-06 17:24:57,441 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-1:job-24
= [ 85b0ca6d-7586-4225-a9aa-b772ea99e3a8 ]) VM state transitted from :Starting to Stopped
with event: OperationFailedvm's original host id: null new host id: null host id before state
transition: null
2013-08-06 17:24:57,447 DEBUG [cloud.api.ApiServlet] (catalina-exec-14:null) ===END===  10.216.133.64
-- GET  command=queryAsyncJobResult&jobId=85b0ca6d-7586-4225-a9aa-b772ea99e3a8&response=json&sessionkey=kxfo3DkYEzBv44N8ynxFwuMn6bE%3D&_=1375835530701
2013-08-06 17:24:57,460 DEBUG [cloud.vm.UserVmManagerImpl] (Job-Executor-1:job-24 = [ 85b0ca6d-7586-4225-a9aa-b772ea99e3a8
]) Destroying vm VM[User|187ff69f-e0d9-4364-bef6-86b59690b152] as it failed to create on Host
with Id:null
2013-08-06 17:24:58,427 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-1:job-24
= [ 85b0ca6d-7586-4225-a9aa-b772ea99e3a8 ]) VM state transitted from :Stopped to Error with
event: OperationFailedToErrorvm's original host id: null new host id: null host id before
state transition: null
2013-08-06 17:24:58,446 WARN  [apache.cloudstack.alerts] (Job-Executor-1:job-24 = [ 85b0ca6d-7586-4225-a9aa-b772ea99e3a8
])  alertType:: 8 // dataCenterId:: 1 // podId:: 1 // clusterId:: null // message:: Failed
to deploy Vm with Id: 14, on Host with Id: null
2013-08-06 17:24:58,476 INFO  [user.vm.DeployVMCmd] (Job-Executor-1:job-24 = [ 85b0ca6d-7586-4225-a9aa-b772ea99e3a8
]) com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment
for VM[User|187ff69f-e0d9-4364-bef6-86b59690b152]Scope=interface com.cloud.dc.DataCenter;
id=1
2013-08-06 17:24:58,476 INFO  [user.vm.DeployVMCmd] (Job-Executor-1:job-24 = [ 85b0ca6d-7586-4225-a9aa-b772ea99e3a8
]) Unable to create a deployment for VM[User|187ff69f-e0d9-4364-bef6-86b59690b152]
com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for
VM[User|187ff69f-e0d9-4364-bef6-86b59690b152]Scope=interface com.cloud.dc.DataCenter; id=1
        at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:888)
        at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:624)
        at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:237)
        at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3408)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2968)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2954)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
        at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
2013-08-06 17:24:58,477 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-1:job-24 = [
85b0ca6d-7586-4225-a9aa-b772ea99e3a8 ]) Complete async job-24 = [ 85b0ca6d-7586-4225-a9aa-b772ea99e3a8
], jobStatus: 2, resultCode: 530, result: Error Code: 533 Error text: Unable to create a deployment
for VM[User|187ff69f-e0d9-4364-bef6-86b59690b152]
2013-08-06 17:24:59,563 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-9:null)
SeqA 3-788: Processing Seq 3-788:  { Cmd , MgmtId: -1, via: 3, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":2,"_loadInfo":"{\n
 \"connections\": []\n}","wait":0}}] }
2013-08-06 17:24:59,765 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-9:null)
SeqA 3-788: Sending Seq 3-788:  { Ans: , MgmtId: 7264031213276, via: 3, Ver: v1, Flags: 100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }


............







--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message