cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "angeline shen (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (CLOUDSTACK-5665) XEN patch/hotfix certification - XS 6.0.2 XS602E030 patch installation fail
Date Fri, 21 Feb 2014 01:56:19 GMT

     [ https://issues.apache.org/jira/browse/CLOUDSTACK-5665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

angeline shen closed CLOUDSTACK-5665.
-------------------------------------


> XEN patch/hotfix certification - XS 6.0.2 XS602E030 patch installation fail
> ---------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-5665
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5665
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Management Server
>    Affects Versions: 4.2.1
>            Reporter: angeline shen
>            Priority: Critical
>             Fix For: 4.2.1
>
>         Attachments: management-server.log.gz
>
>
>  
>  XEN hotfix installation procedure
> 1. Hosts master slave installed with
>     XS 6.0.2 base + Hotfix XS602E001 to XS602E029 + CSPs
> 2. Basic zone with cluster of above master & slave hosts
>     security group default with NO security rules
>     Deploy two VMs
> 3. MS: master host put to maintenance mode - VMs migrate to slave host
> 4. MS: unmanage cluster
> 5. master host: install hotfix XS602E030 + CSP , reboot master
> 6. MS: master host cancel maintenance mode
> 7. MS: manage cluster
> 8. MS: slave host put to maintenance mode, VMs FAIL migrate to master 
>    Unable to complete hotfix  installation
> 2013-12-26 16:51:25,307 INFO  [xen.resource.CitrixResourceBase] (DirectAgent-56:null)
VM does not exist on XenServerc6c75483-b35b-4a7c-9e2d-2274e34dd437
> 2013-12-26 16:51:25,307 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-56:null)
Seq 1-265617550: Response Received: 
> 2013-12-26 16:51:25,307 DEBUG [agent.transport.Request] (DirectAgent-56:null) Seq 1-265617550:
Processing:  { Ans: , MgmtId: 7692017993539, via: 1, Ver: v1, Flags: 110, [{"com.cloud.agent.api.StopAnswer":{"result":true,"details":"VM
does not exist","wait":0}}] }
> 2013-12-26 16:51:25,307 DEBUG [agent.manager.AgentAttache] (DirectAgent-56:null) Seq
1-265617550: Unable to find listener.
> 2013-12-26 16:51:25,307 DEBUG [agent.manager.AgentAttache] (DirectAgent-56:null) Seq
1-265617550: No more commands found
> 2013-12-26 16:51:25,313 DEBUG [vm.dao.VMInstanceDaoImpl] (HA-Worker-3:work-19) Unable
to update VM[DomainRouter|r-4-VM]: DB Data={Host=4; State=Running; updated=13; time=Thu Dec
26 16:51:25 PST 2013} New Data: {Host=4; State=Stopping; updated=12; time=Thu Dec 26 16:51:25
PST 2013} Stale Data: {Host=4; State=Running; updated=11; time=Thu Dec 26 16:51:22 PST 2013}
> 2013-12-26 16:51:25,313 DEBUG [cloud.vm.VirtualMachineManagerImpl] (HA-Worker-3:work-19)
Unable to stop VM due to VM is being operated on.
> 2013-12-26 16:51:25,313 WARN  [cloud.ha.HighAvailabilityManagerImpl] (HA-Worker-3:work-19)
Unable to migrate vm from 4
> 2013-12-26 16:51:25,314 DEBUG [cloud.resource.ResourceManagerImpl] (HA-Worker-3:work-19)
No next resource state for host 4 while current state is ErrorInMaintenance with event UnableToMigrate
> com.cloud.utils.fsm.NoTransitionException: No next resource state found for current state
=ErrorInMaintenance event =UnableToMigrate
>         at com.cloud.resource.ResourceManagerImpl.resourceStateTransitTo(ResourceManagerImpl.java:1178)
>         at com.cloud.resource.ResourceManagerImpl.maintenanceFailed(ResourceManagerImpl.java:2313)
>         at com.cloud.ha.HighAvailabilityManagerImpl.migrate(HighAvailabilityManagerImpl.java:602)
>         at com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.run(HighAvailabilityManagerImpl.java:858)
> 2013-12-26 16:51:25,314 INFO  [cloud.ha.HighAvailabilityManagerImpl] (HA-Worker-3:work-19)
Completed HAWork[19-Migration-4-Running-Migrating]
> 2013-12-26 16:51:25,324 WARN  [xen.resource.CitrixResourceBase] (DirectAgent-57:null)
Task failed! Task record:                 uuid: 38ce6cd4-9ab2-7e46-4d10-dafe25a83084
>            nameLabel: Async.VM.pool_migrate
>      nameDescription: 
>    allowedOperations: []
>    currentOperations: {}
>              created: Thu Dec 26 16:55:03 PST 2013
>             finished: Thu Dec 26 16:55:03 PST 2013
>               status: failure
>           residentOn: com.xensource.xenapi.Host@40561b9c
>             progress: 1.0
>                 type: <none/>
>               result: 
>            errorInfo: [VM_REQUIRES_SR, OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a,
OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]
>          otherConfig: {}
>            subtaskOf: com.xensource.xenapi.Task@aaf13f6f
>             subtasks: []
> 2013-12-26 16:51:25,332 WARN  [xen.resource.CitrixResourceBase] (DirectAgent-57:null)
Unable to migrate VM(s-1-VM) from host(d044a044-1a30-4c01-ae83-11d2631a4ffe) due to Task failed!
Task record:                 uuid: 38ce6cd4-9ab2-7e46-4d10-dafe25a83084
>            nameLabel: Async.VM.pool_migrate
>      nameDescription: 
>    allowedOperations: []
>    currentOperations: {}
>              created: Thu Dec 26 16:55:03 PST 2013
>             finished: Thu Dec 26 16:55:03 PST 2013
>               status: failure
>           residentOn: com.xensource.xenapi.Host@40561b9c
>             progress: 1.0
>                 type: <none/>
>               result: 
>            errorInfo: [VM_REQUIRES_SR, OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a,
OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]
>          otherConfig: {}
>            subtaskOf: com.xensource.xenapi.Task@aaf13f6f
>             subtasks: []
> Task failed! Task record:                 uuid: 38ce6cd4-9ab2-7e46-4d10-dafe25a83084
>            nameLabel: Async.VM.pool_migrate
>      nameDescription: 
> :
>          residentOn: com.xensource.xenapi.Host@40561b9c
>             progress: 1.0
>                 type: <none/>
>               result: 
>            errorInfo: [VM_REQUIRES_SR, OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a,
OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]
>          otherConfig: {}
>            subtaskOf: com.xensource.xenapi.Task@aaf13f6f
>             subtasks: []
>         at com.cloud.hypervisor.xen.resource.CitrixResourceBase.migrateVM(CitrixResourceBase.java:3789)
>         at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:3346)
>         at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:511)
>         at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:59)
>         at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2013-12-26 16:51:25,342 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-57:null)
6. The VM s-1-VM is in Migrating state
> 2013-12-26 16:51:25,342 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-57:null)
Seq 4-961806364: Response Received: 
> 2013-12-26 16:51:25,342 DEBUG [agent.transport.Request] (DirectAgent-57:null) Seq 4-961806364:
Processing:  { Ans: , MgmtId: 7692017993539, via: 4, Ver: v1, Flags: 110, [{"com.cloud.agent.api.MigrateAnswer":{"result":false,"details":"Catch
Exception com.cloud.utils.exception.CloudRuntimeException: Migration failed due to com.cloud.utils.exception.CloudRuntimeException:
Unable to migrate VM(s-1-VM) from host(d044a044-1a30-4c01-ae83-11d2631a4ffe) due to Task failed!
Task record:                 uuid: 38ce6cd4-9ab2-7e46-4d10-dafe25a83084\n           nameLabel:
Async.VM.pool_migrate\n     nameDescription: \n   allowedOperations: []\n   currentOperations:
{}\n             created: Thu Dec 26 16:55:03 PST 2013\n            finished: Thu Dec 26 16:55:03
PST 2013\n              status: failure\n          residentOn: com.xensource.xenapi.Host@40561b9c\n
           progress: 1.0\n                type: <none/>\n              result: \n  
        errorInfo: [VM_REQUIRES_SR, OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a, OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]\n
        otherConfig: {}\n           subtaskOf: com.xensource.xenapi.Task@aaf13f6f\n      
     subtasks: []\n","wait":0}}] }
> 2013-12-26 16:51:25,342 DEBUG [agent.transport.Request] (HA-Worker-0:work-16) Seq 4-961806364:
Received:  { Ans: , MgmtId: 7692017993539, via: 4, Ver: v1, Flags: 110, { MigrateAnswer }
}
> 2013-12-26 16:51:25,342 ERROR [cloud.vm.VirtualMachineManagerImpl] (HA-Worker-0:work-16)
Unable to migrate due to Catch Exception com.cloud.utils.exception.CloudRuntimeException:
Migration failed due to com.cloud.utils.exception.CloudRuntimeException: Unable to migrate
VM(s-1-VM) from host(d044a044-1a30-4c01-ae83-11d2631a4ffe) due to Task failed! Task record:
                uuid: 38ce6cd4-9ab2-7e46-4d10-dafe25a83084
>            nameLabel: Async.VM.pool_migrate
>      nameDescription: 
>    allowedOperations: []
>    currentOperations: {}
>              created: Thu Dec 26 16:55:03 PST 2013
>             finished: Thu Dec 26 16:55:03 PST 2013
>               status: failure
>           residentOn: com.xensource.xenapi.Host@40561b9c
>             progress: 1.0
>                 type: <none/>
>               result: 
>            errorInfo: [VM_REQUIRES_SR, OpaqueRef:6e9c09e8-9ed5-1e21-2048-d848a084a15a,
OpaqueRef:eec84d31-9691-1ba4-f7bb-39db42ac8628]
>          otherConfig: {}
>            subtaskOf: com.xensource.xenapi.Task@aaf13f6f
>             subtasks: []
> 2013-12-26 16:51:25,342 INFO  [cloud.vm.VirtualMachineManagerImpl] (HA-Worker-0:work-16)
Migration was unsuccessful.  Cleaning up: VM[SecondaryStorageVm|s-1-VM]
> 2013-12-26 16:51:25,346 DEBUG [agent.manager.AgentAttache] (DirectAgent-57:null) Seq
4-961806364: No more commands found
> 2013-12-26 16:51:25,355 WARN  [apache.cloudstack.alerts] (HA-Worker-0:work-16)  alertType::
17 // dataCenterId:: 1 // podId:: 1 // clusterId:: null // message:: Unable to migrate vm
s-1-VM from host Rack2Host19.lab.vmops.com in zone z1 and pod z1p1
> 2013-12-26 16:51:25,356 DEBUG [cloud.alert.AlertManagerImpl] (HA-Worker-0:work-16) Have
already sent: 1 emails for alert type '17' -- skipping send email
> 2013-12-26 16:51:25,357 DEBUG [agent.manager.AgentAttache] (HA-Worker-0:work-16) Seq
1-265617551: Routed from 7692017993539
> 2013-12-26 16:51:25,358 DEBUG [agent.transport.Request] (HA-Worker-0:work-16) Seq 1-265617551:
Sending  { Cmd , MgmtId: 7692017993539, via: 1, Ver: v1, Flags: 100111, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":true,"vmName":"s-1-VM","wait":0}}]
}
> 2013-12-26 16:51:25,358 DEBUG [agent.transport.Request] (HA-Worker-0:work-16) Seq 1-265617551:
Executing:  { Cmd , MgmtId: 7692017993539, via: 1, Ver: v1, Flags: 100111, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":true,"vmName":"s-1-VM","wait":0}}]
}
> 2013-12-26 16:51:25,358 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-62:null)
Seq 1-265617551: Executing request
> 2013-12-26 16:51:25,368 DEBUG [cloud.capacity.CapacityManagerImpl] (HA-Worker-0:work-16)
VM state transitted from :Migrating to Running with event: OperationFailedvm's original host
id: 4 new host id: 4 host id before state transition: 1
> 2013-12-26 16:51:25,375 DEBUG [cloud.capacity.CapacityManagerImpl] (HA-Worker-0:work-16)
Hosts's actual total CPU: 9044 and CPU after applying overprovisioning: 9044
> 2013-12-26 16:51:25,375 DEBUG [cloud.capacity.CapacityManagerImpl] (HA-Worker-0:work-16)
Hosts's actual total RAM: 15988871808 and RAM after applying overprovisioning: 15988872192
> 2013-12-26 16:51:25,375 DEBUG [cloud.capacity.CapacityManagerImpl] (HA-Worker-0:work-16)
release cpu from host: 1, old used: 500,reserved: 0, actual total: 9044, total with overprovisioning:
9044; new used: 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
> 2013-12-26 16:51:25,376 DEBUG [cloud.capacity.CapacityManagerImpl] (HA-Worker-0:work-16)
release mem from host: 1, old used: 268435456,reserved: 0, total: 15988872192; new used: 0,reserved:0;
movedfromreserved: false,moveToReserveredfalse
> 2013-12-26 16:51:25,385 INFO  [xen.resource.CitrixResourceBase] (DirectAgent-62:null)
VM does not exist on XenServerc6c75483-b35b-4a7c-9e2d-2274e34dd437
> 2013-12-26 16:51:25,385 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-62:null)
Seq 1-265617551: Response Received: 
> 2013-12-26 16:51:25,386 DEBUG [agent.transport.Request] (DirectAgent-62:null) Seq 1-265617551:
Processing:  { Ans: , MgmtId: 7692017993539, via: 1, Ver: v1, Flags: 110, [{"com.cloud.agent.api.StopAnswer":{"result":true,"details":"VM
does not exist","wait":0}}] }
> 2013-12-26 16:51:25,386 DEBUG [agent.manager.AgentAttache] (DirectAgent-62:null) Seq
1-265617551: Unable to find listener.
> 2013-12-26 16:51:25,386 DEBUG [agent.manager.AgentAttache] (DirectAgent-62:null) Seq
1-265617551: No more commands found
> 2013-12-26 16:51:25,390 DEBUG [vm.dao.VMInstanceDaoImpl] (HA-Worker-0:work-16) Unable
to update VM[SecondaryStorageVm|s-1-VM]: DB Data={Host=4; State=Running; updated=13; time=Thu
Dec 26 16:51:25 PST 2013} New Data: {Host=4; State=Stopping; updated=12; time=Thu Dec 26 16:51:25
PST 2013} Stale Data: {Host=4; State=Running; updated=11; time=Thu Dec 26 16:51:23 PST 2013}
> 2013-12-26 16:51:25,390 DEBUG [cloud.vm.VirtualMachineManagerImpl] (HA-Worker-0:work-16)
Unable to stop VM due to VM is being operated on.
> 2013-12-26 16:51:25,390 WARN  [cloud.ha.HighAvailabilityManagerImpl] (HA-Worker-0:work-16)
Unable to migrate vm from 4
> 2013-12-26 16:51:25,391 DEBUG [cloud.resource.ResourceManagerImpl] (HA-Worker-0:work-16)
No next resource state for host 4 while current state is ErrorInMaintenance with event UnableToMigrate
> com.cloud.utils.fsm.NoTransitionException: No next resource state found for current state
=ErrorInMaintenance event =UnableToMigrate
>         at com.cloud.resource.ResourceManagerImpl.resourceStateTransitTo(ResourceManagerImpl.java:1178)
>         at com.cloud.resource.ResourceManagerImpl.maintenanceFailed(ResourceManagerImpl.java:2313)
>         at com.cloud.ha.HighAvailabilityManagerImpl.migrate(HighAvailabilityManagerImpl.java:602)
>         at com.cloud.ha.HighAvailabilityManagerImpl$WorkerThread.run(HighAvailabilityManagerImpl.java:858)
> 2013-12-26 16:51:25,392 INFO  [cloud.ha.HighAvailabilityManagerImpl] (HA-Worker-0:work-16)
Completed HAWork[16-Migration-1-Running-Migrating]
> 2013-12-26 16:51:26,339 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-10:null)
SeqA 2-416: Processing Seq 2-416:  { Cmd , MgmtId: -1, via: 2, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":2,"_loadInfo":"{\n
 \"connections\": []\n}","wait":0}}] }
> 2013-12-26 16:51:26,346 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-10:null)
SeqA 2-416: Sending Seq 2-416:  { Ans: , MgmtId: 7692017993539, via: 2, Ver: v1, Flags: 100010,
[{"com.cloud.agent.ap



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message