cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wei Zhou (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CLOUDSTACK-4405) (Upgrade) Migrate failed between existing hosts and new hosts
Date Tue, 27 Aug 2013 18:11:53 GMT

    [ https://issues.apache.org/jira/browse/CLOUDSTACK-4405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751532#comment-13751532
] 

Wei Zhou commented on CLOUDSTACK-4405:
--------------------------------------

Migration from old hosts to new hosts.


2013-08-19 17:43:39,154 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-5:null) submit
async job-807, details: AsyncJobVO {id:807, userId: 2, accountId: 2, sessionKey: null, instanceType:
None, instanceId: null, cmd: org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd, cmdOriginator:
null, cmdInfo: {"response":"json","sessionkey":"4c28lulCRIdxTr/qWl4mefFeKKM\u003d","virtualmachineid":"1dc955ec-f579-46b4-a309-8f307a3a659f","hostid":"a2c8fd4f-e621-4937-a4fb-0d365c5602b8","ctxUserId":"2","_":"1376927019023","ctxAccountId":"2","ctxStartEventId":"3230"},
cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0, processStatus: 0, resultCode:
0, result: null, initMsid: 90520747364525, completeMsid: null, lastUpdated: null, lastPolled:
null, created: null}
2013-08-19 17:43:39,156 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-23:job-807)
Executing org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd for job-807
2013-08-19 17:43:39,199 INFO  [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-23:job-807)
Migrating VM[User|1dc955ec-f579-46b4-a309-8f307a3a659f] to Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
: Dest[Zone(1)-Pod(1)-Cluster(1)-Host(40)-Storage()]
2013-08-19 17:43:39,216 DEBUG [cloud.storage.StorageManagerImpl] (Job-Executor-23:job-807)
Preparing 1 volumes for VM[User|1dc955ec-f579-46b4-a309-8f307a3a659f]
2013-08-19 17:43:39,241 DEBUG [agent.manager.ClusteredAgentAttache] (Job-Executor-23:job-807)
Seq 40-1658454123: Forwarding Seq 40-1658454123:  { Cmd , MgmtId: 90520747364525, via: 40,
Ver: v1, Flags: 100111, [{"PrepareForMigrationCommand":{"vm":{"id":139,"name":"i-2-139-VM","type":"User","cpus":1,"speed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Ubuntu
10.04 (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":true,"limitCpuUse":false,"vncPassword":"2db63303ec0086ed","params":{},"uuid":"1dc955ec-f579-46b4-a309-8f307a3a659f","disks":[{"id":132,"name":"ROOT-139","mountPoint":"/storage/da-upgr-01-primary","path":"f877348a-42b1-43f5-a392-da99486a3cf3","size":42949672960,"type":"ROOT","storagePoolType":"NetworkFilesystem","storagePoolUuid":"72301816-c94a-3ad4-a1fd-7a1749c56171","deviceId":0}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"uuid":"fc305b98-a282-498b-be3c-ea6230edcaea","ip":"10.11.110.222","netmask":"255.255.255.0","gateway":"10.11.110.254","mac":"06:0a:d4:00:00:36","dns1":"8.8.8.8","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://110","isolationUri":"vlan://110","isSecurityGroupEnabled":true,"name":"cloudbr0"}]},"wait":0}}]
} to 90520735855750
2013-08-19 17:43:39,333 DEBUG [agent.transport.Request] (Job-Executor-23:job-807) Seq 40-1658454123:
Received:  { Ans: , MgmtId: 90520747364525, via: 40, Ver: v1, Flags: 110, { PrepareForMigrationAnswer
} }
2013-08-19 17:43:39,365 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-23:job-807)
VM state transitted from :Running to Migrating with event: MigrationRequestedvm's original
host id: 11 new host id: 40 host id before state transition: 11
2013-08-19 17:43:39,371 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-23:job-807)
Hosts's actual total CPU: 20216 and CPU after applying overprovisioning: 202160
2013-08-19 17:43:39,371 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-23:job-807)
We are allocating VM, increasing the used capacity of this host:40
2013-08-19 17:43:39,371 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-23:job-807)
Current Used CPU: 500 , Free CPU:201660 ,Requested CPU: 500
2013-08-19 17:43:39,371 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-23:job-807)
Current Used RAM: 536870912 , Free RAM:16176271360 ,Requested RAM: 536870912
2013-08-19 17:43:39,371 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-23:job-807)
CPU STATS after allocation: for host: 40, old used: 500, old reserved: 0, actual total: 20216,
total with overprovisioning: 202160; new used:1000, reserved:0; requested cpu:500,alloc_from_last:false
2013-08-19 17:43:39,371 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-23:job-807)
RAM STATS after allocation: for host: 40, old used: 536870912, old reserved: 0, total: 16713142272;
new used: 1073741824, reserved: 0; requested mem: 536870912,alloc_from_last:false
2013-08-19 17:43:39,390 DEBUG [agent.manager.ClusteredAgentAttache] (Job-Executor-23:job-807)
Seq 11-1310281978: Forwarding Seq 11-1310281978:  { Cmd , MgmtId: 90520747364525, via: 11,
Ver: v1, Flags: 100111, [{"MigrateCommand":{"vmName":"i-2-139-VM","destIp":"192.168.103.111","hostGuid":"93ff4a81-0a4c-3842-8b3d-dcb50bb1cb25-LibvirtComputingResource","isWindows":false,"wait":0}}]
} to 90520735855750
2013-08-19 17:43:39,888 DEBUG [agent.transport.Request] (Job-Executor-23:job-807) Seq 11-1310281978:
Received:  { Ans: , MgmtId: 90520747364525, via: 11, Ver: v1, Flags: 110, { MigrateAnswer
} }
2013-08-19 17:43:39,888 ERROR [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-23:job-807)
Unable to migrate due to Cannot get interface MTU on 'cloudVirBr110': No such device
2013-08-19 17:43:39,888 INFO  [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-23:job-807)
Migration was unsuccessful.  Cleaning up: VM[User|1dc955ec-f579-46b4-a309-8f307a3a659f]
2013-08-19 17:43:39,890 DEBUG [cloud.alert.AlertManagerImpl] (Job-Executor-23:job-807) Have
already sent: 1 emails for alert type '16' -- skipping send email
2013-08-19 17:43:39,894 DEBUG [agent.manager.ClusteredAgentAttache] (Job-Executor-23:job-807)
Seq 40-1658454124: Forwarding Seq 40-1658454124:  { Cmd , MgmtId: 90520747364525, via: 40,
Ver: v1, Flags: 100111, [{"StopCommand":{"isProxy":false,"vmName":"i-2-139-VM","wait":0}}]
} to 90520735855750
2013-08-19 17:43:39,908 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-23:job-807)
VM state transitted from :Migrating to Running with event: OperationFailedvm's original host
id: 11 new host id: 11 host id before state transition: 40
2013-08-19 17:43:39,914 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-23:job-807)
Hosts's actual total CPU: 20216 and CPU after applying overprovisioning: 202160
2013-08-19 17:43:39,914 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-23:job-807)
release cpu from host: 40, old used: 1000,reserved: 0, actual total: 20216, total with overprovisioning:
202160; new used: 500,reserved:0; movedfromreserved: false,moveToReserveredfalse
2013-08-19 17:43:39,914 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-23:job-807)
release mem from host: 40, old used: 1073741824,reserved: 0, total: 16713142272; new used:
536870912,reserved:0; movedfromreserved: false,moveToReserveredfalse
2013-08-19 17:43:39,954 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-23:job-807)
Complete async job-807, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text:
Failed to migrate vm
2013-08-19 17:43:42,230 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-4:null) Async
job-807 completed

                
> (Upgrade) Migrate failed between existing hosts and new hosts
> -------------------------------------------------------------
>
>                 Key: CLOUDSTACK-4405
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4405
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>    Affects Versions: 4.1.0, 4.2.0
>         Environment: CS 4.1
>            Reporter: Wei Zhou
>            Assignee: Kishan Kavala
>            Priority: Blocker
>             Fix For: 4.1.1, 4.2.0, 4.2.1
>
>
> There are two hosts (cs-kvm001, cs-kvm002) in old 2.2.14 environment .
> After upgrade from 2.2.14 to 4.1, I added two new hosts (cs-kvm003, cs-kvm004).
> The migration between cs-kvm001 and cs-kvm002, or cs-kvm003 and cs-kvm004 succeed.
> However, the migration from cs-kvm001/002 to the new hosts (cs-kvm003, cs-kvm004) failed.
> 2013-08-19 16:57:31,051 DEBUG [kvm.resource.BridgeVifDriver] (agentRequest-Handler-1:null)
nic=[Nic:Guest-10.11.110.231-vlan://110]
> 2013-08-19 16:57:31,051 DEBUG [kvm.resource.BridgeVifDriver] (agentRequest-Handler-1:null)
creating a vlan dev and bridge for guest traffic per traffic label cloudbr0
> 2013-08-19 16:57:31,051 DEBUG [utils.script.Script] (agentRequest-Handler-1:null) Executing:
/bin/bash -c brctl show | grep cloudVirBr110
> 2013-08-19 16:57:31,063 DEBUG [utils.script.Script] (agentRequest-Handler-1:null) Exit
value is 1
> 2013-08-19 16:57:31,063 DEBUG [utils.script.Script] (agentRequest-Handler-1:null)
> 2013-08-19 16:57:31,063 DEBUG [kvm.resource.BridgeVifDriver] (agentRequest-Handler-1:null)
Executing: /usr/share/cloudstack-common/scripts/vm/network/vnet/modifyvlan.sh -v 110 -p em1
-b brem1-110 -o add
> 2013-08-19 16:57:31,121 DEBUG [kvm.resource.BridgeVifDriver] (agentRequest-Handler-1:null)
Execution is successful.
> 2013-08-19 16:57:31,122 DEBUG [kvm.resource.BridgeVifDriver] (agentRequest-Handler-1:null)
Set name-type for VLAN subsystem. Should be visible in /proc/net/vlan/config
> This is because the bridge name on old hosts are cloudVirBr110, and brem1-110 on new
hosts.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message