Return-Path: X-Original-To: apmail-cloudstack-issues-archive@www.apache.org Delivered-To: apmail-cloudstack-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 20C1C1031A for ; Tue, 5 Nov 2013 08:02:20 +0000 (UTC) Received: (qmail 89198 invoked by uid 500); 5 Nov 2013 08:02:19 -0000 Delivered-To: apmail-cloudstack-issues-archive@cloudstack.apache.org Received: (qmail 89017 invoked by uid 500); 5 Nov 2013 08:02:18 -0000 Mailing-List: contact issues-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cloudstack.apache.org Delivered-To: mailing list issues@cloudstack.apache.org Received: (qmail 89000 invoked by uid 500); 5 Nov 2013 08:02:18 -0000 Delivered-To: apmail-incubator-cloudstack-issues@incubator.apache.org Received: (qmail 88996 invoked by uid 99); 5 Nov 2013 08:02:18 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 05 Nov 2013 08:02:18 +0000 Date: Tue, 5 Nov 2013 08:02:18 +0000 (UTC) From: "ASF subversion and git services (JIRA)" To: cloudstack-issues@incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (CLOUDSTACK-5028) Instance is failing to start after releasing the primary storage from maintenance mode MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CLOUDSTACK-5028?page=3Dcom.atla= ssian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId= =3D13813738#comment-13813738 ]=20 ASF subversion and git services commented on CLOUDSTACK-5028: ------------------------------------------------------------- Commit 2f459404934ee5ce78499828881a379ae180eee7 in branch refs/heads/master= from [~likithas] [ https://git-wip-us.apache.org/repos/asf?p=3Dcloudstack.git;h=3D2f45940 ] CLOUDSTACK-5028. Vmware instance fails to start when the chain_info of any = volume that belongs to the VM is longer than 255 characters. If the VM has snapshots then the chain_info of a volume can be longer than = 255 characters. Increasing the column length of chain_info in VolumeVO to match the maximum= length of type text(db schema type) > Instance is failing to start after releasing the primary storage from ma= intenance mode > -------------------------------------------------------------------------= -------------- > > Key: CLOUDSTACK-5028 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-502= 8 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the defa= ult.)=20 > Components: Storage Controller, VMware > Affects Versions: 4.2.1 > Reporter: Sailaja Mada > Assignee: Likitha Shetty > Priority: Critical > Fix For: 4.2.1 > > Attachments: startvmlogs.rar > > > Steps: > 1. Configure two Adv zones using VMWARE hypervisor > 2. Configuration with Zone wide primary storages > 3. Deploy VM using a user account=20 > 4. Move the Primary storage into maintenance=20 > 5. With this VM got into stopped state > 6. Release the Storage into maintenance=20 > 7. Now start the VM. > Observation: > Instance is failing to start after releasing the primary storage from ma= intenance mode > 2013-11-03 19:14:30,652 DEBUG [agent.manager.DirectAgentAttache] (DirectA= gent-330:null) Seq 9-894186938: Executing request > 2013-11-03 19:14:30,665 INFO [vmware.resource.VmwareResource] (DirectAge= nt-330:10.102.192.18) Executing resource StartCommand: {"vm":{"id":24,"name= ":"i-5-24-VM","bootloader":"HVM","type":"User","cpus":1,"minSpeed":200,"max= Speed":200,"minRam":536870912,"maxRam":536870912,"hostName":"sailajaVM1","a= rch":"x86_64","os":"CentOS 5.3 (64-bit)","bootArgs":"","rebootOnCrash":fals= e,"enableHA":true,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vnc= Password":"e5e3f536d1a3d5cc","params":{"nicAdapter":"E1000","vmware.reserve= .cpu":"false","nestedVirtualizationFlag":"false","Message.ReservedCapacityF= reed.Flag":"false","rootDiskController":"ide","vmware.reserve.mem":"false"}= ,"uuid":"1ec83c40-e0dc-48ad-90d8-f4000c0dfe91","disks":[{"data":{"org.apach= e.cloudstack.storage.to.VolumeObjectTO":{"uuid":"1ffa6e41-00cc-47d5-958c-f8= 305ac06af4","volumeType":"DATADISK","dataStore":{"org.apache.cloudstack.sto= rage.to.PrimaryDataStoreTO":{"uuid":"b33e996a-444e-3685-9070-0865067454c4",= "id":6,"poolType":"NetworkFilesystem","host":"10.102.192.100","path":"/cpg_= vol/sailaja/sailajaps2","port":2049}},"name":"DATA-24","size":5368709120,"p= ath":"a933eb3fa28a473ab5e28b99f5f2607e","volumeId":37,"vmName":"i-5-24-VM",= "accountId":5,"chainInfo":"{\"diskDeviceBusName\":\"scsi0:0\",\"diskChain\"= :[\"[7f18caf5397a340a934ed37c558aee2b] i-5-24-VM/8d463fcfe15c43dea0cef44748= 64552e-000004.vmdk\",\"[7f18caf5397a340a934ed37c558aee2b] i-5-24-VM/8d463fc= fe15c43dea0cef4474864552e-000003.vmdk\",\"[7f18caf5397a340a934ed37c5","form= at":"OVA","id":37,"hypervisorType":"VMware"}},"diskSeq":1,"type":"DATADISK"= },{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"91d3f= ff2-cc90-4122-b076-06588fa33401","volumeType":"ROOT","dataStore":{"org.apac= he.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7f18caf5-397a-340a-93= 4e-d37c558aee2b","id":5,"poolType":"NetworkFilesystem","host":"10.102.192.1= 00","path":"/cpg_vol/sailaja/sailajaps1","port":2049}},"name":"ROOT-24","si= ze":2147483648,"path":"ROOT-24","volumeId":38,"vmName":"i-5-24-VM","account= Id":5,"format":"OVA","id":38,"hypervisorType":"VMware"}},"diskSeq":0,"type"= :"ROOT"},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"uui= d":"257add99-2031-4ad8-b633-979f3812b1cd","id":200,"format":"ISO","accountI= d":1,"hvm":true,"displayText":"VMware Tools Installer ISO","name":"vmware-t= ools.iso","guestOsType":"CentOS 4.5 (32-bit)","hypervisorType":"VMware"}},"= diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defa= ultNic":true,"uuid":"f3987dbb-7e7e-4a50-8dc1-761b6bdd2f41","ip":"10.1.1.79"= ,"netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:3a:5d:00:01","= dns1":"10.103.128.15","broadcastType":"Vlan","type":"Guest","broadcastUri":= "vlan://777","isolationUri":"vlan://777","isSecurityGroupEnabled":false}]},= "hostIp":"10.102.192.18","executeInSequence":true,"wait":0} > 2013-11-03 19:14:30,747 DEBUG [vmware.mo.HostMO] (DirectAgent-330:10.102.= 192.18) find VM i-5-24-VM on host > 2013-11-03 19:14:30,747 INFO [vmware.mo.HostMO] (DirectAgent-330:10.102.= 192.18) VM i-5-24-VM not found in host cache > 2013-11-03 19:14:30,747 DEBUG [vmware.mo.HostMO] (DirectAgent-330:10.102.= 192.18) load VM cache on host > 2013-11-03 19:14:30,769 DEBUG [cloud.api.ApiServlet] (catalina-exec-25:nu= ll) =3D=3D=3DSTART=3D=3D=3D 10.104.255.45 -- GET command=3DqueryAsyncJobR= esult&jobId=3Dfc31a268-ae48-49c7-8908-35c52ee1b922&response=3Djson&sessionk= ey=3DktQfh1aD0%2Ffcgyias7vCxMP6WIs%3D&_=3D1383486601934 > 2013-11-03 19:14:30,774 INFO [vmware.resource.VmwareResource] (DirectAge= nt-330:10.102.192.18) VM i-5-24-VM already exists, tear down devices for re= configuration > 2013-11-03 19:14:30,806 DEBUG [cloud.api.ApiServlet] (catalina-exec-25:nu= ll) =3D=3D=3DEND=3D=3D=3D 10.104.255.45 -- GET command=3DqueryAsyncJobRes= ult&jobId=3Dfc31a268-ae48-49c7-8908-35c52ee1b922&response=3Djson&sessionkey= =3DktQfh1aD0%2Ffcgyias7vCxMP6WIs%3D&_=3D1383486601934 > 2013-11-03 19:14:31,194 DEBUG [vmware.resource.VmwareResource] (DirectAge= nt-330:10.102.192.18) Prepare ISO volume at existing device {"key":3000,"de= viceInfo":{"label":"CD/DVD drive 1","summary":"Remote device /usr/lib/vmwar= e/isoimages/linux.iso"},"backing":{"exclusive":false,"deviceName":""},"conn= ectable":{"startConnected":true,"allowGuestControl":false,"connected":true}= ,"controllerKey":200,"unitNumber":0} > 2013-11-03 19:14:31,194 INFO [vmware.resource.VmwareResource] (DirectAge= nt-330:10.102.192.18) Found existing disk info from volume path: ROOT-24 > 2013-11-03 19:14:31,195 INFO [vmware.resource.VmwareResource] (DirectAge= nt-330:10.102.192.18) Chose disk controller based on existing information: = ide0:1 > 2013-11-03 19:14:31,195 WARN [vmware.resource.VmwareResource] (DirectAge= nt-330:10.102.192.18) StartCommand failed due to Exception: com.google.gson= .JsonSyntaxException > Message: com.google.gson.stream.MalformedJsonException: Unterminated stri= ng near > com.google.gson.JsonSyntaxException: com.google.gson.stream.MalformedJson= Exception: Unterminated string near > at com.google.gson.Streams.parse(Streams.java:51) > at com.google.gson.Gson.fromJson(Gson.java:497) > at com.google.gson.Gson.fromJson(Gson.java:467) > at com.google.gson.Gson.fromJson(Gson.java:417) > at com.google.gson.Gson.fromJson(Gson.java:389) > at com.cloud.hypervisor.vmware.resource.VmwareResource.getMatchin= gExistingDisk(VmwareResource.java:3280) > at com.cloud.hypervisor.vmware.resource.VmwareResource.execute(Vm= wareResource.java:2855) > at com.cloud.hypervisor.vmware.resource.VmwareResource.executeReq= uest(VmwareResource.java:513) > at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgen= tAttache.java:186) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.= java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:= 334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutu= reTask.access$101(ScheduledThreadPoolExecutor.java:165) > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutu= reTask.run(ScheduledThreadPoolExecutor.java:266) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolEx= ecutor.java:1110) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolE= xecutor.java:603) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.= java:603) > at java.lang.Thread.run(Thread.java:679) > Caused by: com.google.gson.stream.MalformedJsonException: Unterminated st= ring near > at com.google.gson.stream.JsonReader.syntaxError(JsonReader.java:= 1111) > at com.google.gson.stream.JsonReader.nextString(JsonReader.java:9= 67) > at com.google.gson.stream.JsonReader.nextValue(JsonReader.java:80= 2) > at com.google.gson.stream.JsonReader.nextInArray(JsonReader.java:= 706) > at com.google.gson.stream.JsonReader.quickPeek(JsonReader.java:37= 3) > at com.google.gson.stream.JsonReader.hasNext(JsonReader.java:332) > at com.google.gson.Streams.parseRecursive(Streams.java:74) > at com.google.gson.Streams.parseRecursive(Streams.java:83) > at com.google.gson.Streams.parse(Streams.java:40) > ... 16 more > 2013-11-03 19:14:31,199 DEBUG [agent.manager.DirectAgentAttache] (DirectA= gent-330:null) Seq 9-894186938: Response Received: > 2013-11-03 19:14:31,201 DEBUG [agent.transport.Request] (DirectAgent-330:= null) Seq 9-894186938: Processing: { Ans: , MgmtId: 94838926819810, via: 9= , Ver: v1, Flags: 110, [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":24,"= name":"i-5-24-VM","bootloader":"HVM","type":"User","cpus":1,"minSpeed":200,= "maxSpeed":200,"minRam":536870912,"maxRam":536870912,"hostName":"sailajaVM1= ","arch":"x86_64","os":"CentOS 5.3 (64-bit)","bootArgs":"","rebootOnCrash":= false,"enableHA":true,"limitCpuUse":false,"enableDynamicallyScaleVm":false,= "vncPassword":"e5e3f536d1a3d5cc","params":{"nicAdapter":"E1000","vmware.res= erve.cpu":"false","nestedVirtualizationFlag":"false","Message.ReservedCapac= ityFreed.Flag":"false","rootDiskController":"ide","vmware.reserve.mem":"fal= se"},"uuid":"1ec83c40-e0dc-48ad-90d8-f4000c0dfe91","disks":[{"data":{"org.a= pache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"1ffa6e41-00cc-47d5-958= c-f8305ac06af4","volumeType":"DATADISK","dataStore":{"org.apache.cloudstack= .storage.to.PrimaryDataStoreTO":{"uuid":"b33e996a-444e-3685-9070-0865067454= c4","id":6,"poolType":"NetworkFilesystem","host":"10.102.192.100","path":"/= cpg_vol/sailaja/sailajaps2","port":2049}},"name":"DATA-24","size":536870912= 0,"path":"a933eb3fa28a473ab5e28b99f5f2607e","volumeId":37,"vmName":"i-5-24-= VM","accountId":5,"chainInfo":"{\"diskDeviceBusName\":\"scsi0:0\",\"diskCha= in\":[\"[7f18caf5397a340a934ed37c558aee2b] i-5-24-VM/8d463fcfe15c43dea0cef4= 474864552e-000004.vmdk\",\"[7f18caf5397a340a934ed37c558aee2b] i-5-24-VM/8d4= 63fcfe15c43dea0cef4474864552e-000003.vmdk\",\"[7f18caf5397a340a934ed37c5","= format":"OVA","id":37,"hypervisorType":"VMware"}},"diskSeq":1,"type":"DATAD= ISK"},{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"9= 1d3fff2-cc90-4122-b076-06588fa33401","volumeType":"ROOT","dataStore":{"org.= apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"7f18caf5-397a-340= a-934e-d37c558aee2b","id":5,"poolType":"NetworkFilesystem","host":"10.102.1= 92.100","path":"/cpg_vol/sailaja/sailajaps1","port":2049}},"name":"ROOT-24"= ,"size":2147483648,"path":"ROOT-24","volumeId":38,"vmName":"i-5-24-VM","acc= ountId":5,"format":"OVA","id":38,"hypervisorType":"VMware"}},"diskSeq":0,"t= ype":"ROOT"},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{= "uuid":"257add99-2031-4ad8-b633-979f3812b1cd","id":200,"format":"ISO","acco= untId":1,"hvm":true,"displayText":"VMware Tools Installer ISO","name":"vmwa= re-tools.iso","guestOsType":"CentOS 4.5 (32-bit)","hypervisorType":"VMware"= }},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"= defaultNic":true,"uuid":"f3987dbb-7e7e-4a50-8dc1-761b6bdd2f41","ip":"10.1.1= .79","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:3a:5d:00:0= 1","dns1":"10.103.128.15","broadcastType":"Vlan","type":"Guest","broadcastU= ri":"vlan://777","isolationUri":"vlan://777","isSecurityGroupEnabled":false= }]},"result":false,"details":"StartCommand failed due to Exception: com.goo= gle.gson.JsonSyntaxException\nMessage: com.google.gson.stream.MalformedJson= Exception: Unterminated string near \n","wait":0}}] } > 2013-11-03 19:14:31,201 DEBUG [agent.transport.Request] (Job-Executor-27:= job-165 =3D [ fc31a268-ae48-49c7-8908-35c52ee1b922 ]) Seq 9-894186938: Rece= ived: { Ans: , MgmtId: 94838926819810, via: 9, Ver: v1, Flags: 110, { Star= tAnswer } } > 2013-11-03 19:14:31,204 DEBUG [agent.manager.AgentAttache] (DirectAgent-3= 30:null) Seq 9-894186938: No more commands found > 2013-11-03 19:14:31,208 INFO [cloud.vm.VirtualMachineManagerImpl] (Job-E= xecutor-27:job-165 =3D [ fc31a268-ae48-49c7-8908-35c52ee1b922 ]) Unable to = start VM on Host[-9-Routing] due to StartCommand failed due to Exception: c= om.google.gson.JsonSyntaxException > Message: com.google.gson.stream.MalformedJsonException: Unterminated stri= ng near > 2013-11-03 19:14:31,219 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-E= xecutor-27:job-165 =3D [ fc31a268-ae48-49c7-8908-35c52ee1b922 ]) Cleaning u= p resources for the vm VM[User|sailajaVM1] in Starting state > 2013-11-03 19:14:31,221 DEBUG [agent.transport.Request] (Job-Executor-27:= job-165 =3D [ fc31a268-ae48-49c7-8908-35c52ee1b922 ]) Seq 9-894186939: Send= ing { Cmd , MgmtId: 94838926819810, via: 9, Ver: v1, Flags: 100111, [{"com= .cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":true,"vm= Name":"i-5-24-VM","wait":0}}] } > 2013-11-03 19:14:31,222 DEBUG [agent.transport.Request] (Job-Executor-27:= job-165 =3D [ fc31a268-ae48-49c7-8908-35c52ee1b922 ]) Seq 9-894186939: Exec= uting: { Cmd , MgmtId: 94838926819810, via: 9, Ver: v1, Flags: 100111, [{"= com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":true,= "vmName":"i-5-24-VM","wait":0}}] } > 2013-11-03 19:14:31,244 DEBUG [agent.manager.DirectAgentAttache] (DirectA= gent-184:null) Seq 9-894186939: Executing request > 2013-11-03 19:14:31,245 INFO [vmware.resource.VmwareResource] (DirectAge= nt-184:10.102.192.18) Executing resource StopCommand: {"isProxy":false,"exe= cuteInSequence":true,"vmName":"i-5-24-VM","wait":0} > 2013-11-03 19:14:31,264 DEBUG [vmware.mo.HostMO] (DirectAgent-184:10.102.= 192.18) find VM i-5-24-VM on host > 2013-11-03 19:14:31,264 INFO [vmware.mo.HostMO] (DirectAgent-184:10.102.= 192.18) VM i-5-24-VM not found in host cache > 2013-11-03 19:14:31,264 DEBUG [vmware.mo.HostMO] (DirectAgent-184:10.102.= 192.18) load VM cache on host > 2013-11-03 19:14:31,322 INFO [vmware.resource.VmwareResource] (DirectAge= nt-184:10.102.192.18) VM i-5-24-VM is already in stopped state -- This message was sent by Atlassian JIRA (v6.1#6144)