cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abhinandan Prateek (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CLOUDSTACK-9503) The router script times out resulting in failure of deployment
Date Tue, 20 Sep 2016 04:16:20 GMT
Abhinandan Prateek created CLOUDSTACK-9503:
----------------------------------------------

             Summary: The router script times out resulting in failure of deployment
                 Key: CLOUDSTACK-9503
                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9503
             Project: CloudStack
          Issue Type: Bug
      Security Level: Public (Anyone can view this level - this is the default.)
          Components: Virtual Router
    Affects Versions: 4.9.0
         Environment: KVM, Xen
            Reporter: Abhinandan Prateek
            Assignee: Abhinandan Prateek
             Fix For: 4.9.1.0


When starting the virtual router in a shared network in advance zone the scripts on router
time out. This happen as there are several sub-commands that are consolidated in a single
command. The default timeout of 2 minutes is short.



2016-09-09 00:02:54,336 DEBUG [c.c.a.t.Request] (Work-Job-Executor-110:ctx-e8089ec7 job-5135/job-5137
ctx-c3a8da18) (logid:8aedea66) Seq 341-7107524636920779325: Sending { Cmd , MgmtId: 345048838161,
via: 341(tckkwjih-pbhpv032.cloud_platform.kddi.ne.jp), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StartCommand":{"vm":{"id":3445,"name":"r-3445-VM","type":"DomainRouter","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":268435456,"maxRam":268435456,"arch":"x86_64","os":"Debian
GNU/Linux 6(64-bit)","platformEmulator":"Debian GNU/Linux 6","bootArgs":" template=domP name=r-3445-VM
eth0ip=198.18.177.3 eth0mask=255.255.255.224 gateway=198.18.177.1 cidrsize=27 dhcprange=198.18.177.1
eth1ip=169.254.3.32 eth1mask=255.255.0.0 type=dhcpsrvr disable_rp_filter=true dns1=106.162.226.43
dns2= ip6dns1= ip6dns2= useextdns=true","enableHA":true,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"sQTkDM6LZudYM83ofRrR6g","params":{},"uuid":"bd7cadca-3d7b-4e41-83de-8a35d7176337","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"ee8b9d8f-00ca-4f10-98e3-202852bab484","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"530740c2-1cf0-3017-ae74-509de53d4bf4","id":335,"poolType":"NetworkFilesystem","host":"10.194.2.31","path":"/tckktky2-vnx5300_01/tckkwjih-pEbsv02Middle02","port":2049,"url":"NetworkFilesystem://10.194.2.31/tckktky2-vnx5300_01/tckkwjih-pEbsv02Middle02/?ROLE=Primary&STOREUUID=530740c2-1cf0-3017-ae74-509de53d4bf4"}},"name":"ROOT-3445","size":3145728000,"path":"ee8b9d8f-00ca-4f10-98e3-202852bab484","volumeId":47049,"vmName":"r-3445-VM","accountId":1,"format":"QCOW2","provisioningType":"THIN","id":47049,"deviceId":0,"bytesReadRate":0,"bytesWriteRate":0,"iopsReadRate":0,"iopsWriteRate":0,"cacheMode":"NONE","hypervisorType":"KVM"}},"diskSeq":0,"path":"ee8b9d8f-00ca-4f10-98e3-202852bab484","type":"ROOT","_details":{"managed":"false","storagePort":"2049","storageHost":"10.194.2.31","volumeSize":"3145728000"}}],"nics":[
{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"pxeDisable":true,"nicUuid":"bb4d7609-269f-47ba-941b-cb58817f8af5","uuid":"21b19e01-6f0f-4bde-b0fb-58fae768b6e3","ip":"198.18.177.3","netmask":"255.255.255.224","gateway":"198.18.177.1","mac":"06:57:8a:00:0f:9e","dns1":"106.162.226.43","dns2":"","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://3301","isolationUri":"vlan://3301","isSecurityGroupEnabled":false}
,
{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"746c3fdb-6ae6-4a52-a1bd-c47dca70853a","uuid":"cba84c68-0b08-4966-957a-62e10a5a9711","ip":"169.254.3.32","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:03:20","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false}
]},"hostIp":"10.194.3.32","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.3.32","port":3922,"interval":6,"retries":100,"name":"r-3445-VM","wait":0}},{"com.cloud.agent.api.GetDomRVersionCmd":{"accessDetails":
{"router.name":"r-3445-VM","router.ip":"169.254.3.32"}
,"wait":0}},{},{"com.cloud.agent.api.routing.AggregationControlCommand":{"action":"Start","accessDetails":
{"router.guest.ip":"198.18.177.3","router.name":"r-3445-VM","router.ip":"169.254.3.32"}
,"wait":0}},{"com.cloud.agent.api.routing.SetMonitorServiceCommand":{"services":[
{"id":0,"service":"dhcp","processname":"dnsmasq","serviceName":"dnsmasq","servicePath":"/var/run/dnsmasq/dnsmasq.pid","pidFile":"/var/run/dnsmasq/dnsmasq.pid","isDefault":false}
,
{"id":0,"service":"ssh","processname":"sshd","serviceName":"ssh","servicePath":"/var/run/sshd.pid","pidFile":"/var/run/sshd.pid","isDefault":true}
,
{"id":0,"service":"webserver","processname":"apache2","serviceName":"apache2","servicePath":"/var/run/apache2.pid","pidFile":"/var/run/apache2.pid","isDefault":true}
],"accessDetails":
{"router.name":"r-3445-VM","router.ip":"169.254.3.32","router.guest.ip":"198.18.177.3"}
,"wait":0}},{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"06:df:56:00:0f:a2","vmIpAddress":"198.18.177.7","vmName":"p-cl-vCenter-win-02-M00000001","defaultRouter":"198.18.177.1","duid":"00:03:00:01:06:df:56:00:0f:a2","isDefault":false,"executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.guest.ip":"198.18.177.3","router.ip":"169.254.3.32","router.name":"r-3445-VM"}
,"wait":0}},{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"06:cb:80:00:0f:a6","vmIpAddress":"198.18.177.11","vmName":"p-cl-KVM-linux-03-M00000001","defaultRouter":"0.0.0.0","duid":"00:03:00:01:06:cb:80:00:0f:a6","isDefault":false,"executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.guest.ip":"198.18.177.3","router.ip":"169.254.3.32","router.name":"r-3445-VM"}
,"wait":0}},{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"06:e7:4c:00:0f:a9","vmIpAddress":"198.18.177.14","vmName":"p-cl-KVM-win-03-M00000001","defaultRouter":"198.18.177.1","duid":"00:03:00:01:06:e7:4c:00:0f:a9","isDefault":false,"executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.guest.ip":"198.18.177.3","router.ip":"169.254.3.32","router.name":"r-3445-VM"}
,"wait":0}},{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"06:c9:80:00:0f:aa","vmIpAddress":"198.18.177.15","vmName":"p-cl-test-vol-KVM-M00000001","defaultRouter":"0.0.0.0","duid":"00:03:00:01:06:c9:80:00:0f:aa","isDefault":false,"executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.guest.ip":"198.18.177.3","router.ip":"169.254.3.32","router.name":"r-3445-VM"}
,"wait":0}},{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"06:7c:0c:00:0f:ab","vmIpAddress":"198.18.177.16","vmName":"v-cl-vmtest-KVM-R-0100-M00000001","defaultRouter":"0.0.0.0","duid":"00:03:00:01:06:7c:0c:00:0f:ab","isDefault":false,"executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.guest.ip":"198.18.177.3","router.ip":"169.254.3.32","router.name":"r-3445-VM"}
,"wait":0}},{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"06:78:26:00:0f:ad","vmIpAddress":"198.18.177.18","vmName":"v-cl-vmtest-KVM-W-0300-M00000001","defaultRouter":"198.18.177.1","duid":"00:03:00:01:06:78:26:00:0f:ad","isDefault":false,"executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.guest.ip":"198.18.177.3","router.ip":"169.254.3.32","router.name":"r-3445-VM"}
,"wait":0}},{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"06:e0:28:00:0f:ae","vmIpAddress":"198.18.177.19","vmName":"p-cl-vmtest-KVM-W-0400-M00000001","defaultRouter":"198.18.177.1","duid":"00:03:00:01:06:e0:28:00:0f:ae","isDefault":false,"executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.guest.ip":"198.18.177.3","router.ip":"169.254.3.32","router.name":"r-3445-VM"}
,"wait":0}},{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"06:24:a4:00:0f:b0","vmIpAddress":"198.18.177.21","vmName":"cl-vmtest-KVM-001","defaultRouter":"0.0.0.0","duid":"00:03:00:01:06:24:a4:00:0f:b0","isDefault":false,"executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.guest.ip":"198.18.177.3","router.ip":"169.254.3.32","router.name":"r-3445-VM"}
,"wait":0}},{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"06:98:76:00:0f:a4","vmIpAddress":"198.18.177.9","vmName":"p-cl-KVM-linux-01-M00000001","defaultRouter":"0.0.0.0","duid":"00:03:00:01:06:98:76:00:0f:a4","isDefault":false,"executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.guest.ip":"198.18.177.3","router.ip":"169.254.3.32","router.name":"r-3445-VM"}
,"wait":0}},{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"06:0a:b0:00:0f:b4","vmIpAddress":"198.18.177.25","vmName":"cl-vmtest-VMware-001","defaultRouter":"0.0.0.0","duid":"00:03:00:01:06:0a:b0:00:0f:b4","isDefault":false,"executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.guest.ip":"198.18.177.3","router.ip":"169.254.3.32","router.name":"r-3445-VM"}
,"wait":0}},{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"06:f4:e0:00:0f:ac","vmIpAddress":"198.18.177.17","vmName":"p-cl-vmtest-VM-R-0600-M00000001","defaultRouter":"0.0.0.0","duid":"00:03:00:01:06:f4:e0:00:0f:ac","isDefault":false,"executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.guest.ip":"198.18.177.3","router.ip":"169.254.3.32","router.name":"r-3445-VM"}
,"wait":0}},{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"06:c4:ca:00:0f:af","vmIpAddress":"198.18.177.20","vmName":"p-cl-vmtest-VM-W-0800-M00000001","defaultRouter":"198.18.177.1","duid":"00:03:00:01:06:c4:ca:00:0f:af","isDefault":false,"executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.guest.ip":"198.18.177.3","router.ip":"169.254.3.32","router.name":"r-3445-VM"}
,"wait":0}},{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"06:0a:ec:00:0f:b3","vmIpAddress":"198.18.177.24","vmName":"p-cl-reg-test-VMware-1-M00000001","defaultRouter":"0.0.0.0","duid":"00:03:00:01:06:0a:ec:00:0f:b3","isDefault":false,"executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.guest.ip":"198.18.177.3","router.ip":"169.254.3.32","router.name":"r-3445-VM"}
,"wait":0}},{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"06:09:ec:00:0f:b2","vmIpAddress":"198.18.177.23","vmName":"p-cl-reg-test-VMware-2-M00000001","defaultRouter":"198.18.177.1","duid":"00:03:00:01:06:09:ec:00:0f:b2","isDefault":false,"executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.guest.ip":"198.18.177.3","router.ip":"169.254.3.32","router.name":"r-3445-VM"}
,"wait":0}},{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"06:6a:90:00:0f:b5","vmIpAddress":"198.18.177.26","vmName":"v-cl-Autoreg-0133-M00000001","defaultRouter":"0.0.0.0","duid":"00:03:00:01:06:6a:90:00:0f:b5","isDefault":false,"executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.guest.ip":"198.18.177.3","router.ip":"169.254.3.32","router.name":"r-3445-VM"}
,"wait":0}},{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"06:88:d0:00:0f:b6","vmIpAddress":"198.18.177.27","vmName":"v-cl-Autoreg-0133-002-M00000001","defaultRouter":"0.0.0.0","duid":"00:03:00:01:06:88:d0:00:0f:b6","isDefault":false,"executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.guest.ip":"198.18.177.3","router.ip":"169.254.3.32","router.name":"r-3445-VM"}
,"wait":0}},{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"06:52:c8:00:0f:a0","vmIpAddress":"198.18.177.5","vmName":"v-mm0906a-M00000001","defaultRouter":"0.0.0.0","duid":"00:03:00:01:06:52:c8:00:0f:a0","isDefault":false,"executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.guest.ip":"198.18.177.3","router.ip":"169.254.3.32","router.name":"r-3445-VM"}
,"wait":0}},{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"06:26:16:00:0f:b7","vmIpAddress":"198.18.177.28","vmName":"p-mm0906c-M00000001","defaultRouter":"0.0.0.0","duid":"00:03:00:01:06:26:16:00:0f:b7","isDefault":false,"executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.guest.ip":"198.18.177.3","router.ip":"169.254.3.32","router.name":"r-3445-VM"}
,"wait":0}},{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"06:e2:1e:00:0f:a3","vmIpAddress":"198.18.177.8","vmName":"v-mm0906bb-M00000001","defaultRouter":"198.18.177.1","duid":"00:03:00:01:06:e2:1e:00:0f:a3","isDefault":false,"executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.guest.ip":"198.18.177.3","router.ip":"169.254.3.32","router.name":"r-3445-VM"}
,"wait":0}},{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"06:76:f8:00:0f:b9","vmIpAddress":"198.18.177.30","vmName":"v-cl-kvm-v5-M00000001","defaultRouter":"0.0.0.0","duid":"00:03:00:01:06:76:f8:00:0f:b9","isDefault":false,"executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.guest.ip":"198.18.177.3","router.ip":"169.254.3.32","router.name":"r-3445-VM"}
,"wait":0}},{"com.cloud.agent.api.routing.VmDataCommand":{"vmIpAddress":"198.18.177.7","vmName":"p-cl-vCenter-win-02-M00000001","executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.name":"r-3445-VM","router.ip":"169.254.3.32","router.guest.ip":"198.18.177.3"}
,"wait":0}},{"com.cloud.agent.api.routing.VmDataCommand":{"vmIpAddress":"198.18.177.11","vmName":"p-cl-KVM-linux-03-M00000001","executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.name":"r-3445-VM","router.ip":"169.254.3.32","router.guest.ip":"198.18.177.3"}
,"wait":0}},{"com.cloud.agent.api.routing.VmDataCommand":{"vmIpAddress":"198.18.177.14","vmName":"p-cl-KVM-win-03-M00000001","executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.name":"r-3445-VM","router.ip":"169.254.3.32","router.guest.ip":"198.18.177.3"}
,"wait":0}},{"com.cloud.agent.api.routing.VmDataCommand":{"vmIpAddress":"198.18.177.15","vmName":"p-cl-test-vol-KVM-M00000001","executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.name":"r-3445-VM","router.ip":"169.254.3.32","router.guest.ip":"198.18.177.3"}
,"wait":0}},{"com.cloud.agent.api.routing.VmDataCommand":{"vmIpAddress":"198.18.177.16","vmName":"v-cl-vmtest-KVM-R-0100-M00000001","executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.name":"r-3445-VM","router.ip":"169.254.3.32","router.guest.ip":"198.18.177.3"}
,"wait":0}},{"com.cloud.agent.api.routing.VmDataCommand":{"vmIpAddress":"198.18.177.18","vmName":"v-cl-vmtest-KVM-W-0300-M00000001","executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.name":"r-3445-VM","router.ip":"169.254.3.32","router.guest.ip":"198.18.177.3"}
,"wait":0}},{"com.cloud.agent.api.routing.VmDataCommand":{"vmIpAddress":"198.18.177.19","vmName":"p-cl-vmtest-KVM-W-0400-M00000001","executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.name":"r-3445-VM","router.ip":"169.254.3.32","router.guest.ip":"198.18.177.3"}
,"wait":0}},{"com.cloud.agent.api.routing.VmDataCommand":{"vmIpAddress":"198.18.177.21","vmName":"cl-vmtest-KVM-001","executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.name":"r-3445-VM","router.ip":"169.254.3.32","router.guest.ip":"198.18.177.3"}
,"wait":0}},{"com.cloud.agent.api.routing.VmDataCommand":{"vmIpAddress":"198.18.177.9","vmName":"p-cl-KVM-linux-01-M00000001","executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.name":"r-3445-VM","router.ip":"169.254.3.32","router.guest.ip":"198.18.177.3"}
,"wait":0}},{"com.cloud.agent.api.routing.VmDataCommand":{"vmIpAddress":"198.18.177.25","vmName":"cl-vmtest-VMware-001","executeInSequence":false,"accessDetails":
{"zone.network.type":"Advanced","router.name":"r-3445-VM","router.ip":"169.254.3.32","router.guest.ip":"198.18.177.3"}
,"wait":0}},{"com.cloud.agent.api.routing.VmDataCommand":{"vmIpAddress":"198.18.177.17","vmName":"p-cl-vmtest-VM-R-0600-M00000001","executeInSequence":false,"accessDetails":{"zone.network.type":"Advanced","router.name":"r-3445-VM","router.ip":"169
—
2016-09-09 00:05:38,933 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-efbed4df) (logid:7d56e29d)
Task (job-5135) has been pending for 165 seconds
2016-09-09 00:05:38,933 WARN [o.a.c.f.j.i.AsyncJobMonitor] (Timer-1:ctx-efbed4df) (logid:7d56e29d)
Task (job-5137) has been pending for 165 seconds
—
2016-09-09 00:06:25,007 DEBUG [c.c.a.t.Request] (Work-Job-Executor-110:ctx-e8089ec7 job-5135/job-5137
ctx-c3a8da18) (logid:8aedea66) Seq 341-7107524636920779325: Received: { Ans: , MgmtId: 345048838161,
via: 341(tckkwjih-pbhpv032.cloud_platform.kddi.ne.jp), Ver: v1, Flags: 10,
{ StartAnswer, CheckSshAnswer, GetDomRVersionAnswer, NetworkUsageAnswer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer }
}
—
2016-09-09 00:06:25,012 WARN [o.a.c.alerts] (Work-Job-Executor-110:ctx-e8089ec7 job-5135/job-5137
ctx-c3a8da18) (logid:8aedea66) alertType:: 9 // dataCenterId:: 1 // podId:: 1 // clusterId::
null // message:: Command: com.cloud.agent.api.Command failed while starting virtual router
—
2016-09-09 00:06:25,016 ERROR [c.c.n.r.VirtualNetworkApplianceManagerImpl] (Work-Job-Executor-110:ctx-e8089ec7
job-5135/job-5137 ctx-c3a8da18) (logid:8aedea66) process hasn't exited
2016-09-09 00:06:25,016 WARN [c.c.n.r.VirtualNetworkApplianceManagerImpl] (Work-Job-Executor-110:ctx-e8089ec7
job-5135/job-5137 ctx-c3a8da18) (logid:8aedea66) Command: com.cloud.agent.api.Command failed
while starting virtual router
2016-09-09 00:06:25,016 INFO [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-110:ctx-e8089ec7
job-5135/job-5137 ctx-c3a8da18) (logid:8aedea66) The guru did not like the answers so stopping
VM[DomainRouter|r-3445-VM]
—



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message