cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kelven Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CLOUDSTACK-4376) [Automation] [vmware] Failed to deploy VPC router in parallel deployment scenario
Date Tue, 20 Aug 2013 22:07:53 GMT

    [ https://issues.apache.org/jira/browse/CLOUDSTACK-4376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13745512#comment-13745512
] 

Kelven Yang commented on CLOUDSTACK-4376:
-----------------------------------------

Thanks Rayee to reproduce the problem so that I can have access the environment. It turned
out to be a problem in VR startup script under VPC mode. There exists even script syntax error!
Don't know how the code can be checked-in in the first place
                
> [Automation] [vmware] Failed to deploy VPC router in parallel deployment scenario
> ---------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-4376
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4376
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: API, Automation
>    Affects Versions: 4.2.0
>         Environment: Vmware
> 4.2 build
> Automation enviroenment
>            Reporter: Rayees Namathponnan
>            Assignee: Kelven Yang
>            Priority: Blocker
>             Fix For: 4.2.0
>
>         Attachments: CLOUDSTACK-4376.rar, management-server.zip
>
>
> In Automation environment VPC test cases failed during VPC router deployment; we are
running 5 tests in parallel
> Attached log for more information; here the example for router " r-278", 
> In below log, after creating router r-278,  trying ssh connection, its failed 
> 2013-08-16 07:57:43,108 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-71:job-908
= [ 4d43324e-92f7-4a6e-87c9-691459396cc6 ]) Asking BareMetalUserdata to prepare for Nic[536-278-4893da91-fa2d-4675-ad74-25c1d83f9ce5-10.223.250.170]
> 2013-08-16 07:57:43,110 DEBUG [cloud.storage.VolumeManagerImpl] (Job-Executor-71:job-908
= [ 4d43324e-92f7-4a6e-87c9-691459396cc6 ]) Checking if we need to prepare 1 volumes f
> or VM[DomainRouter|r-278-TestVM]
> 2013-08-16 07:57:43,165 DEBUG [agent.transport.Request] (Job-Executor-71:job-908 = [
4d43324e-92f7-4a6e-87c9-691459396cc6 ]) Seq 3-806750713: Executing:  { Cmd , MgmtId: 90928106758026,
via: 3, Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"772bc80f5a773aafa713c41017b167e4","origUrl":"http://download.cloud.com/templates/4.2/systemvmtemplate-4.2-vh7.ova","uuid":"4dff74ac-0633-11e3-a733-52b2d980df8a","id":8,"format":"OVA","accountId":1,"checksum":"8fde62b1089e5844a9cd3b9b953f9596","hvm":false,"displayText":"SystemVM
Template (vSphere)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"16c8ee2e-3f26-3062-8878-b7d6c53c5797","id":2,"poolType":"NetworkFilesystem","host":"10.223.110.232","path":"/export/home/automation/SC-CLOUD-QA03/primary2","port":2049}},"name":"routing-8","hypervisorType":"VMware"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"95cb6848-a917-4012-b1e0-faceba7cb374","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"16c8ee2e-3f26-3062-8878-b7d6c53c5797","id":2,"poolType":"NetworkFilesystem","host":"10.223.110.232","path":"/export/home/automation/SC-CLOUD-QA03/primary2","port":2049}},"name":"ROOT-278","size":2097152000,"volumeId":316,"vmName":"r-278-TestVM","accountId":183,"format":"OVA","id":316,"hypervisorType":"None"}},"executeInSequence":false,"wait":0}}]
}
> 2013-08-16 07:57:43,167 DEBUG [cloud.api.ApiServlet] (catalina-exec-7:null) ===END===
 10.223.240.195 -- GET  signature=dabpu%2BmE%2B3ToMpP%2BHLlBT0IFPEI%3D&apiKey=i4-XPSPRUUp7UQhz7PkZ7AsThtQ1d4LTEqLlFgg8yQxdtPaekrzmnPfjMyMUoCsd_AUFYGSzr7AqZAji5kNR1w&command=queryAsyncJobResult&response=json&jobid=fb6335e4-f30d-49eb-8b66-a5fd684363dd
> 2013-08-16 07:57:43,167 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-311:null)
Seq 3-806750713: Executing request
> 2013-08-16 07:58:05,152 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-71:job-908
= [ 4d43324e-92f7-4a6e-87c9-691459396cc6 ]) Boot Args for VM[DomainRouter|r-278-TestVM]: 
vpccidr=10.0.0.1/24 domain=csb7auto.advanced dns1=8.8.8.8 template=domP name=r-278-TestVM
eth0ip=10.223.250.170 eth0mask=255.255.255.192 mgmtcidr=10.223.49.192/26 localgw=10.223.250.129
type=vpcrouter disable_rp_filter=true extra_pubnics=2
> 2013-08-16 07:58:05,214 DEBUG [network.router.VpcVirtualNetworkApplianceManagerImpl]
(Job-Executor-71:job-908 = [ 4d43324e-92f7-4a6e-87c9-691459396cc6 ]) Found 0 static routes
to apply as a part of vpc route VM[DomainRouter|r-278-TestVM] start
> 2013-08-16 08:22:35,131 INFO  [storage.resource.VmwareStorageLayoutHelper] (DirectAgent-61:10.223.250.130)
sync [16c8ee2e3f2630628878b7d6c53c5797] ROOT-278-delta.vmdk->[16c8ee
> 2e3f2630628878b7d6c53c5797] r-278-TestVM/ROOT-278-delta.vmdk
> 2013-08-16 08:22:35,383 INFO  [storage.resource.VmwareStorageLayoutHelper] (DirectAgent-61:10.223.250.130)
sync [16c8ee2e3f2630628878b7d6c53c5797] ROOT-278.vmdk->[16c8ee2e3f26
> 30628878b7d6c53c5797] r-278-TestVM/ROOT-278.vmdk
> 2013-08-16 08:22:35,654 DEBUG [vmware.resource.VmwareResource] (DirectAgent-61:10.223.250.130)
Prepare volume at new device {"capacityInKB":0,"key":-2,"backing":{"diskMode":"p
> ersistent","fileName":"[16c8ee2e3f2630628878b7d6c53c5797] r-278-TestVM/ROOT-278.vmdk","datastore":{"value":"datastore-26","type":"Datastore"}},"connectable":{"startConnected":
> true,"allowGuestControl":false,"connected":true},"controllerKey":1000,"unitNumber":0}
> 2013-08-16 08:22:35,654 INFO  [vmware.resource.VmwareResource] (DirectAgent-61:10.223.250.130)
Prepare NIC device based on NicTO: {"deviceId":0,"networkRateMbps":-1,"defaultNi
> c":false,"uuid":"1313c17c-eabe-4fb4-b672-e7d3f10713b4","ip":"10.223.250.170","netmask":"255.255.255.192","gateway":"10.223.250.129","mac":"02:00:75:9b:00:8d","broadcastType":"
> Native","type":"Control","isSecurityGroupEnabled":false}
> 2013-08-16 08:22:35,658 INFO  [vmware.resource.VmwareResource] (DirectAgent-61:10.223.250.130)
Prepare network on vmwaresvs P[null:untagged] with name prefix: cloud.private
> 2013-08-16 08:22:35,658 INFO  [vmware.mo.HypervisorHostHelper] (DirectAgent-61:10.223.250.130)
Detected vswitch name as undefined. Defaulting to vSwitch0
> 2013-08-16 08:22:35,749 INFO  [vmware.mo.HypervisorHostHelper] (DirectAgent-61:10.223.250.130)
Network cloud.private.untagged.0.1-vSwitch0 is ready on vSwitch vSwitch0
> 2013-08-16 08:22:35,749 INFO  [vmware.resource.VmwareResource] (DirectAgent-61:10.223.250.130)
Preparing NIC device on network cloud.private.untagged.0.1-vSwitch0
> 2013-08-16 08:22:35,749 DEBUG [vmware.resource.VmwareResource] (DirectAgent-61:10.223.250.130)
Prepare NIC at new device {"operation":"ADD","device":{"addressType":"Manual","m
> acAddress":"02:00:75:9b:00:8d","key":-3,"backing":{"network":{"value":"network-24","type":"Network"},"deviceName":"cloud.private.untagged.0.1-vSwitch0"},"connectable":{"startC
> onnected":true,"allowGuestControl":true,"connected":true},"unitNumber":0}}
> 2013-08-16 08:22:35,749 DEBUG [vmware.mo.HostMO] (DirectAgent-61:10.223.250.130) find
VM r-278-TestVM on host
> 2013-08-16 08:22:35,749 DEBUG [vmware.mo.HostMO] (DirectAgent-61:10.223.250.130) VM r-278-TestVM
found in host cache
> 2013-08-16 08:22:35,788 INFO  [vmware.resource.VmwareResource] (DirectAgent-61:10.223.250.130)
Configure VNC port for VM r-278-TestVM, port: 5950, host: 10.223.250.130
> 2013-08-16 08:48:32,246 ERROR [vmware.resource.VmwareResource] (DirectAgent-61:10.223.250.130)
Can not ping System vm r-278-TestVMdue to:Unable to connect
> 2013-08-16 08:48:32,246 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-61:null)
Seq 3-806750716: Cancelling because one of the answers is false and it is stop on error.
> 2013-08-16 08:48:32,246 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-61:null)
Seq 3-806750716: Response Received:
> 2013-08-16 08:48:32,247 DEBUG [agent.transport.Request] (DirectAgent-61:null) Seq 3-806750716:
Processing:  { Ans: , MgmtId: 90928106758026, via: 3, Ver: v1, Flags: 110, [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":278,"name":"r-278-TestVM","bootloader":"HVM","type":"DomainRouter","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":134217728,"maxRam":134217728,"hostName":"r-278-TestVM","arch":"i686","os":"Debian
GNU/Linux 5.0 (32-bit)","bootArgs":" vpccidr=10.0.0.1/24 domain=csb7auto.advanced dns1=8.8.8.8
template=domP name=r-278-TestVM eth0ip=10.223.250.170 eth0mask=255.255.255.192 mgmtcidr=10.223.49.192/26
localgw=10.223.250.129 type=vpcrouter disable_rp_filter=true extra_pubnics=2 nic_macs=02:00:75:9b:00:8d","rebootOnCrash":false,"enableHA":true,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"4af860eb17e29895","params":{"nicAdapter":"E1000"},"uuid":"881e4c0b-264d-40ea-8859-a4195f67ca11","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"95cb6848-a917-4012-b1e0-faceba7cb374","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"16c8ee2e-3f26-3062-8878-b7d6c53c5797","id":2,"poolType":"NetworkFilesystem","host":"10.223.110.232","path":"/export/home/automation/SC-CLOUD-QA03/primary2","port":2049}},"name":"ROOT-278","size":2097152000,"path":"ROOT-278","volumeId":316,"vmName":"r-278-TestVM","accountId":183,"format":"OVA","id":316,"hypervisorType":"VMware"}},"diskSeq":0,"type":"ROOT"}],"nics":[{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"uuid":"1313c17c-eabe-4fb4-b672-e7d3f10713b4","ip":"10.223.250.170","netmask":"255.255.255.192","gateway":"10.223.250.129","mac":"02:00:75:9b:00:8d","broadcastType":"Native","type":"Control","isSecurityGroupEnabled":false}]},"result":true,"wait":0}},{"com.cloud.agent.api.check.CheckSshAnswer":{"result":false,"details":"Can
not ping System vm r-278-TestVMdue to:Unable to connect","wait":0}}] }
> 2013-08-16 08:48:32,261 WARN  [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-71:job-908
= [ 4d43324e-92f7-4a6e-87c9-691459396cc6 ]) Unable to ssh to the VM: Can not ping System vm
r-278-TestVMdue to:Unable to connect
> 2013-08-16 08:48:32,262 INFO  [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-71:job-908
= [ 4d43324e-92f7-4a6e-87c9-691459396cc6 ]) The guru did not like the answers so stopping
VM[DomainRouter|r-278-TestVM]
>  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message