Return-Path: X-Original-To: apmail-cloudstack-issues-archive@www.apache.org Delivered-To: apmail-cloudstack-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B94BB108AB for ; Wed, 28 Aug 2013 00:09:54 +0000 (UTC) Received: (qmail 44715 invoked by uid 500); 28 Aug 2013 00:09:53 -0000 Delivered-To: apmail-cloudstack-issues-archive@cloudstack.apache.org Received: (qmail 44693 invoked by uid 500); 28 Aug 2013 00:09:53 -0000 Mailing-List: contact issues-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cloudstack.apache.org Delivered-To: mailing list issues@cloudstack.apache.org Received: (qmail 44658 invoked by uid 500); 28 Aug 2013 00:09:53 -0000 Delivered-To: apmail-incubator-cloudstack-issues@incubator.apache.org Received: (qmail 44649 invoked by uid 99); 28 Aug 2013 00:09:53 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 28 Aug 2013 00:09:53 +0000 Date: Wed, 28 Aug 2013 00:09:53 +0000 (UTC) From: "ASF subversion and git services (JIRA)" To: cloudstack-issues@incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (CLOUDSTACK-4518) unable to start router vm on vmware4.1 after upgrade; hitting exception MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CLOUDSTACK-4518?page=3Dcom.atla= ssian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId= =3D13751935#comment-13751935 ]=20 ASF subversion and git services commented on CLOUDSTACK-4518: ------------------------------------------------------------- Commit f2c8469c599a262150a2ed8638a32ff78319ddac in branch refs/heads/4.2 fr= om [~kelveny] [ https://git-wip-us.apache.org/repos/asf?p=3Dcloudstack.git;h=3Df2c8469 ] CLOUDSTACK-4518: make folder sync resilience with possible runtime situatio= n (cherry picked from commit 8326dfb76f3072dab63243ef2cd62fa3e7b7c688) Signed-off-by: animesh =20 > unable to start router vm on vmware4.1 after upgrade; hitting exception > ----------------------------------------------------------------------- > > Key: CLOUDSTACK-4518 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-451= 8 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the defa= ult.)=20 > Components: Upgrade, VMware > Affects Versions: 4.2.1 > Environment: 3.0.4 patch1 to 4.2 upgrade with exi4.1 host > Reporter: shweta agarwal > Assignee: Kelven Yang > Priority: Blocker > Fix For: 4.2.1 > > Attachments: management-server-router.log.tar.gz > > > id and upgrade on 3.0.4 patch1 to 4.2 . > Having two advance zone > 1 with EXI 4.1 > second with xen > With vmware zone i stopped/start router vm once . It passed > but from second time onward if i stop router vm , start of router vm fail= s with exception > router vm fails to start > Getting following exception in Ms log :=20 > 2013-08-27 14:11:38,901 DEBUG [cloud.api.ApiServlet] (catalina-exec-7:nul= l) =3D=3D=3DEND=3D=3D=3D 10.146.0.13 -- GET command=3DqueryAsyncJobResult= &jobId=3D6cbdc727-6f37-4311-bf68-c494ae8d1606&response=3Djson&sessionkey=3D= JfGwnKwdSVyjkqkZFWtNSAK0JKU%3D&_=3D1377607652726 > 2013-08-27 14:11:39,034 WARN [vmware.resource.VmwareResource] (DirectAge= nt-302:10.147.40.27) StartCommand failed due to Exception: java.lang.Except= ion > Message: Unable to find ROOT-28.vmdk in datastore: d9a91dba343e38dcbbe169= a3f57fd335 > java.lang.Exception: Unable to find ROOT-28.vmdk in datastore: d9a91dba34= 3e38dcbbe169a3f57fd335 > at com.cloud.storage.resource.VmwareStorageLayoutHelper.syncVolum= eToRootFolder(VmwareStorageLayoutHelper.java:142) > at com.cloud.storage.resource.VmwareStorageLayoutHelper.syncVolum= eToVmDefaultFolder(VmwareStorageLayoutHelper.java:115) > at com.cloud.hypervisor.vmware.resource.VmwareResource.syncDiskCh= ain(VmwareResource.java:3012) > at com.cloud.hypervisor.vmware.resource.VmwareResource.execute(Vm= wareResource.java:2858) > at com.cloud.hypervisor.vmware.resource.VmwareResource.executeReq= uest(VmwareResource.java:515) > at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgen= tAttache.java:186) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.= java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:= 334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutu= reTask.access$101(ScheduledThreadPoolExecutor.java:165) > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutu= reTask.run(ScheduledThreadPoolExecutor.java:266) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolEx= ecutor.java:1146) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolE= xecutor.java:615) > at java.lang.Thread.run(Thread.java:679) > 2013-08-27 14:11:39,038 DEBUG [agent.manager.DirectAgentAttache] (DirectA= gent-302:null) Seq 5-982581486: Cancelling because one of the answers is fa= lse and it is stop on error. > 2013-08-27 14:11:39,038 DEBUG [agent.manager.DirectAgentAttache] (DirectA= gent-302:null) Seq 5-982581486: Response Received: > 2013-08-27 14:11:39,034 WARN [vmware.resource.VmwareResource] (DirectAge= nt-302:10.147.40.27) StartCommand failed due to Exception: java.lang.Except= ion > Message: Unable to find ROOT-28.vmdk in datastore: d9a91dba343e38dcbbe169= a3f57fd335 > java.lang.Exception: Unable to find ROOT-28.vmdk in datastore: d9a91dba34= 3e38dcbbe169a3f57fd335 > at com.cloud.storage.resource.VmwareStorageLayoutHelper.syncVolum= eToRootFolder(VmwareStorageLayoutHelper.java:142) > at com.cloud.storage.resource.VmwareStorageLayoutHelper.syncVolum= eToVmDefaultFolder(VmwareStorageLayoutHelper.java:115) > at com.cloud.hypervisor.vmware.resource.VmwareResource.syncDiskCh= ain(VmwareResource.java:3012) > at com.cloud.hypervisor.vmware.resource.VmwareResource.execute(Vm= wareResource.java:2858) > at com.cloud.hypervisor.vmware.resource.VmwareResource.executeReq= uest(VmwareResource.java:515) > at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgen= tAttache.java:186) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.= java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:= 334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutu= reTask.access$101(ScheduledThreadPoolExecutor.java:165) > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutu= reTask.run(ScheduledThreadPoolExecutor.java:266) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolEx= ecutor.java:1146) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolE= xecutor.java:615) > at java.lang.Thread.run(Thread.java:679) > 2013-08-27 14:11:39,038 DEBUG [agent.manager.DirectAgentAttache] (DirectA= gent-302:null) Seq 5-982581486: Cancelling because one of the answers is fa= lse and it is stop on error. > 2013-08-27 14:11:39,038 DEBUG [agent.manager.DirectAgentAttache] (DirectA= gent-302:null) Seq 5-982581486: Response Received: > 2013-08-27 14:11:39,039 DEBUG [agent.transport.Request] (DirectAgent-302:= null) Seq 5-982581486: Processing: { Ans: , MgmtId: 7217726095403, via: 5,= Ver: v1, Flags: 110, [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":28,"n= ame":"r-28-VM","bootloader":"HVM","type":"DomainRouter","cpus":1,"minSpeed"= :500,"maxSpeed":500,"minRam":134217728,"maxRam":134217728,"hostName":"r-28-= VM","arch":"x86_64","os":"Debian GNU/Linux 5.0 (32-bit)","bootArgs":" templ= ate=3DdomP name=3Dr-28-VM eth2ip=3D10.147.51.119 eth2mask=3D255.255.255.0 g= ateway=3D10.147.51.1 eth0ip=3D10.1.1.1 eth0mask=3D255.255.255.0 domain=3Dcs= 2cloud.internal dhcprange=3D10.1.1.1 eth1ip=3D10.147.41.100 eth1mask=3D255.= 255.254.0 mgmtcidr=3D10.147.38.0/24 localgw=3D10.147.40.1 type=3Drouter dis= able_rp_filter=3Dtrue extra_pubnics=3D2 dns1=3D10.103.128.16 nic_macs=3D02:= 00:30:b7:00:02|02:00:7b:f7:00:1c|06:c7:30:00:00:1e|02:00:0f:93:00:09|02:00:= 0f:48:00:0a","rebootOnCrash":false,"enableHA":true,"limitCpuUse":false,"ena= bleDynamicallyScaleVm":false,"vncPassword":"b1d85f74cf74c7f5","params":{"ni= cAdapter":"E1000"},"uuid":"da6d1b87-40c0-4a2a-b8e1-9768a98f3ab1","disks":[{= "data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"84d9cfc9= -432e-4101-bb7b-ea707a6d5744","volumeType":"ROOT","dataStore":{"org.apache.= cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"d9a91dba-343e-38dc-bbe1-= 69a3f57fd335","id":201,"poolType":"NetworkFilesystem","host":"10.147.28.7",= "path":"/export/home/shweta/304patch1.vmware.primary","port":2049}},"name":= "ROOT-28","size":2097152000,"path":"ROOT-28","volumeId":47,"vmName":"r-28-V= M","accountId":2,"chainInfo":"{\"diskDeviceBusName\":\"scsi0:0\",\"diskChai= n\":[\"[d9a91dba343e38dcbbe169a3f57fd335] r-28-VM/ROOT-28.vmdk\"]}","format= ":"OVA","id":47,"hypervisorType":"VMware"}},"diskSeq":0,"type":"ROOT"}],"ni= cs":[{"deviceId":2,"networkRateMbps":200,"defaultNic":true,"uuid":"5f1d98ed= -63b6-4061-b0ae-1067621b5335","ip":"10.147.51.119","netmask":"255.255.255.0= ","gateway":"10.147.51.1","mac":"06:c7:30:00:00:1e","dns1":"10.103.128.16",= "broadcastType":"Vlan","type":"Public","broadcastUri":"vlan://51","isolatio= nUri":"vlan://51","isSecurityGroupEnabled":false},{"deviceId":0,"networkRat= eMbps":200,"defaultNic":false,"uuid":"18ecd99e-131d-4853-8811-4cbf9b1725a9"= ,"ip":"10.1.1.1","netmask":"255.255.255.0","mac":"02:00:30:b7:00:02","dns1"= :"10.103.128.16","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan= ://1058","isolationUri":"vlan://1058","isSecurityGroupEnabled":false},{"dev= iceId":1,"networkRateMbps":-1,"defaultNic":false,"uuid":"e2074834-21d9-4b48= -bb94-ecf362576024","ip":"10.147.41.100","netmask":"255.255.254.0","gateway= ":"10.147.40.1","mac":"02:00:7b:f7:00:1c","broadcastType":"Native","type":"= Control","isSecurityGroupEnabled":false},{"deviceId":3,"networkRateMbps":-1= ,"defaultNic":false,"ip":"0.0.0.0","netmask":"255.255.255.255","gateway":"1= 0.147.51.1","mac":"02:00:0f:93:00:09","dns1":"10.103.128.16","broadcastType= ":"Vlan","type":"Public","broadcastUri":"vlan://51","isolationUri":"vlan://= 51","isSecurityGroupEnabled":false},{"deviceId":4,"networkRateMbps":-1,"def= aultNic":false,"ip":"0.0.0.0","netmask":"255.255.255.255","gateway":"10.147= .51.1","mac":"02:00:0f:48:00:0a","dns1":"10.103.128.16","broadcastType":"Vl= an","type":"Public","broadcastUri":"vlan://51","isolationUri":"vlan://51","= isSecurityGroupEnabled":false}]},"result":false,"details":"StartCommand fai= led due to Exception: java.lang.Exception\nMessage: Unable to find ROOT-28.= vmdk in datastore: d9a91dba343e38dcbbe169a3f57fd335\n","wait":0}}] } > 2013-08-27 14:11:39,039 DEBUG [agent.manager.AgentAttache] (DirectAgent-3= 02:null) Seq 5-982581486: No more commands found > = =20 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrato= rs For more information on JIRA, see: http://www.atlassian.com/software/jira