cloudstack-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mice Xia (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CLOUDSTACK-1687) VMSnapshot: VM Fails to Start after a RevertToVMSnapshot Operation
Date Tue, 19 Mar 2013 04:51:15 GMT

    [ https://issues.apache.org/jira/browse/CLOUDSTACK-1687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13606061#comment-13606061
] 

Mice Xia commented on CLOUDSTACK-1687:
--------------------------------------

Chandan,

i tried the test scenario again with latest master branch (ESXi 5.0), one root disk and two
data disks, no exception/error was observed, which vsphere version did you use?

                
> VMSnapshot: VM Fails to Start after a RevertToVMSnapshot Operation
> ------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-1687
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-1687
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Management Server
>    Affects Versions: 4.2.0
>            Reporter: Chandan Purushothama
>            Assignee: Mice Xia
>            Priority: Blocker
>             Fix For: 4.2.0
>
>         Attachments: management-server.zip
>
>
> ========================
> Steps to Reproduce the Bug:
> ========================
> 1. Deploy Vm.  Attach muiltiple data disks to this VM.
> 2. Log in to the Vm and create few directories and files on the root volume. Create few
more directories on each of  the data disks.
> 3. Stop the VM.
> 4. Create VMSnapshot for this VM with "vm_snapshot_type" set to "Disk"
> 5. Start the VM.
> 6. Delete few of the above created directories and add few more directories on the root
volume.Delete few of the above created directories and add few more directories on the data
disks.
> 7. Stop the VM.
> 8. Revert this Vm to the above created Snapshot.
> 9. Start the VM.
> ===========
> Observations:
> ===========
> *Management Server Log*
> 2013-03-14 16:06:15,703 INFO  [vmware.resource.VmwareResource] (DirectAgent-242:10.223.58.2)
Executing resource StartCommand: {"vm":{"id":9,"name":"i-3-9-VMSNAP","bootloader":"HVM","type":"User","cpus":1,"minSpeed":100,"maxSpeed":100,"minRam":134217728,"maxRam":134217728,"arch":"x86_64","os":"CentOS
5.3 (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"vncPassword":"ec119fd35127c44e","params":{"rootDiskController":"ide","nicAdapter":"E1000"},"uuid":"1c64977b-db4f-415e-98ce-83b53a612493","disks":[{"id":14,"name":"ROOT-9","mountPoint":"/export/home/chandan/asf4-131-170/primary","path":"ROOT-9-14-000001","size":2147483648,"type":"ROOT","storagePoolType":"NetworkFilesystem","storagePoolUuid":"f42a60c2-0940-305b-bb14-23d8b40f42bc","deviceId":0},{"id":15,"name":"DATA-9","mountPoint":"/export/home/chandan/asf4-131-170/primary","path":"c7adbf5e2509484d8ac4959ae53e6146-000001","size":5368709120,"type":"DATADISK","storagePoolType":"NetworkFilesystem","storagePoolUuid":"f42a60c2-0940-305b-bb14-23d8b40f42bc","deviceId":1},{"id":16,"name":"NEWDISK-3","mountPoint":"/export/home/chandan/asf4-131-170/primary","path":"fba51dce35d24c19a224090f3cb597e8-000001","size":5368709120,"type":"DATADISK","storagePoolType":"NetworkFilesystem","storagePoolUuid":"f42a60c2-0940-305b-bb14-23d8b40f42bc","deviceId":2},{"id":9,"name":"CentOS
5.3(64-bit) no GUI (vSphere)","size":0,"type":"ISO","storagePoolType":"ISO","deviceId":3}],"nics":[{"deviceId":0,"networkRateMbps":495,"defaultNic":true,"uuid":"06e76915-e16d-4f60-a311-3a5fbdd7d5ab","ip":"10.1.1.66","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:48:7a:00:07","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://2254","isolationUri":"vlan://2254","isSecurityGroupEnabled":false}]},"hostIp":"10.223.58.2","wait":0}
> 2013-03-14 16:06:15,703 DEBUG [vmware.resource.VmwareResource] (DirectAgent-242:10.223.58.2)
VM i-3-9-VMSNAP will be started with NIC device type: E1000
> 2013-03-14 16:06:15,703 DEBUG [vmware.resource.VmwareResource] (DirectAgent-242:10.223.58.2)
Drop invalid disk option, volumeTO: {"id":9,"name":"CentOS 5.3(64-bit) no GUI (vSphere)","size":0,"type":"ISO","storagePoolType":"ISO","deviceId":3}
> 2013-03-14 16:06:15,723 DEBUG [vmware.mo.HostMO] (DirectAgent-242:10.223.58.2) find VM
i-3-9-VMSNAP on host
> 2013-03-14 16:06:15,723 DEBUG [vmware.mo.HostMO] (DirectAgent-242:10.223.58.2) load VM
cache on host
> 2013-03-14 16:06:15,728 INFO  [vmware.resource.VmwareResource] (DirectAgent-242:10.223.58.2)
VM i-3-9-VMSNAP already exists, tear down devices for reconfiguration
> 2013-03-14 16:06:15,750 DEBUG [vmware.resource.VmwareResource] (DirectAgent-242:10.223.58.2)
Prepare ISO volume at existing device {"key":3000,"deviceInfo":{"label":"CD/DVD drive 1","summary":"Remote
device"},"backing":{"exclusive":false,"deviceName":""},"connectable":{"startConnected":true,"allowGuestControl":false,"connected":true},"controllerKey":200,"unitNumber":0}
> 2013-03-14 16:06:15,753 DEBUG [vmware.resource.VmwareResource] (DirectAgent-242:10.223.58.2)
Prepare volume at new device {"capacityInKB":0,"key":-2,"backing":{"diskMode":"persistent","fileName":"[f42a60c20940305bbb1423d8b40f42bc]
ROOT-9-14-000001.vmdk","datastore":{"value":"datastore-22919","type":"Datastore"}},"connectable":{"startConnected":true,"allowGuestControl":false,"connected":true},"controllerKey":200,"unitNumber":1}
> 2013-03-14 16:06:15,753 DEBUG [vmware.resource.VmwareResource] (DirectAgent-242:10.223.58.2)
Prepare volume at new device {"capacityInKB":0,"key":-3,"backing":{"diskMode":"persistent","fileName":"[f42a60c20940305bbb1423d8b40f42bc]
c7adbf5e2509484d8ac4959ae53e6146-000001.vmdk","datastore":{"value":"datastore-22919","type":"Datastore"}},"connectable":{"startConnected":true,"allowGuestControl":false,"connected":true},"controllerKey":1000,"unitNumber":2}
> 2013-03-14 16:06:15,753 DEBUG [vmware.resource.VmwareResource] (DirectAgent-242:10.223.58.2)
Prepare volume at new device {"capacityInKB":0,"key":-4,"backing":{"diskMode":"persistent","fileName":"[f42a60c20940305bbb1423d8b40f42bc]
fba51dce35d24c19a224090f3cb597e8-000001.vmdk","datastore":{"value":"datastore-22919","type":"Datastore"}},"connectable":{"startConnected":true,"allowGuestControl":false,"connected":true},"controllerKey":1000,"unitNumber":3}
> 2013-03-14 16:06:15,753 INFO  [vmware.resource.VmwareResource] (DirectAgent-242:10.223.58.2)
Prepare NIC device based on NicTO: {"deviceId":0,"networkRateMbps":495,"defaultNic":true,"uuid":"06e76915-e16d-4f60-a311-3a5fbdd7d5ab","ip":"10.1.1.66","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:48:7a:00:07","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://2254","isolationUri":"vlan://2254","isSecurityGroupEnabled":false}
> 2013-03-14 16:06:15,757 INFO  [vmware.resource.VmwareResource] (DirectAgent-242:10.223.58.2)
Prepare network on vmwaresvs P[vSwitch0:untagged] with name prefix: cloud.guest
> 2013-03-14 16:06:15,841 INFO  [vmware.mo.HypervisorHostHelper] (DirectAgent-242:10.223.58.2)
Network cloud.guest.2254.495.1-vSwitch0 is ready on vSwitch vSwitch0
> 2013-03-14 16:06:15,841 INFO  [vmware.resource.VmwareResource] (DirectAgent-242:10.223.58.2)
Preparing NIC device on network cloud.guest.2254.495.1-vSwitch0
> 2013-03-14 16:06:15,841 DEBUG [vmware.resource.VmwareResource] (DirectAgent-242:10.223.58.2)
Prepare NIC at new device {"operation":"ADD","device":{"addressType":"Manual","macAddress":"02:00:48:7a:00:07","key":-5,"backing":{"network":{"value":"network-22944","type":"Network"},"deviceName":"cloud.guest.2254.495.1-vSwitch0"},"connectable":{"startConnected":true,"allowGuestControl":true,"connected":true},"unitNumber":4}}
> 2013-03-14 16:06:15,841 DEBUG [vmware.mo.HostMO] (DirectAgent-242:10.223.58.2) find VM
i-3-9-VMSNAP on host
> 2013-03-14 16:06:15,841 DEBUG [vmware.mo.HostMO] (DirectAgent-242:10.223.58.2) VM i-3-9-VMSNAP
found in host cache
> 2013-03-14 16:06:15,869 INFO  [vmware.resource.VmwareResource] (DirectAgent-242:10.223.58.2)
Configure VNC port for VM i-3-9-VMSNAP, port: 5914, host: 10.223.58.2
> 2013-03-14 16:06:15,923 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:null)
Found 2 routers to update status.
> 2013-03-14 16:06:15,924 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:null)
Found 0 networks to update RvR status.
> 2013-03-14 16:06:16,054 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:null)
Found 2 routers to update status.
> 2013-03-14 16:06:16,055 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:null)
Found 0 networks to update RvR status.
> 2013-03-14 16:06:16,066 WARN  [vmware.resource.VmwareResource] (DirectAgent-242:10.223.58.2)
StartCommand failed due to Exception: java.lang.RuntimeException
> Message: File []/vmfs/volumes/23dd55b9-15861742/ROOT-9-14-000001.vmdk was not found
> java.lang.RuntimeException: File []/vmfs/volumes/23dd55b9-15861742/ROOT-9-14-000001.vmdk
was not found
>         at com.cloud.hypervisor.vmware.util.VmwareClient.waitForTask(VmwareClient.java:291)
>         at com.cloud.hypervisor.vmware.mo.VirtualMachineMO.configureVm(VirtualMachineMO.java:806)
>         at com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:2345)
>         at com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:432)
>         at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2013-03-14 16:06:16,067 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-242:null)
Seq 1-970065218: Response Received:
> 2013-03-14 16:06:16,068 DEBUG [agent.transport.Request] (DirectAgent-242:null) Seq 1-970065218:
Processing:  { Ans: , MgmtId: 7200344900649, via: 1, Ver: v1, Flags: 110, [{"StartAnswer":{"vm":{"id":9,"name":"i-3-9-VMSNAP","bootloader":"HVM","type":"User","cpus":1,"minSpeed":100,"maxSpeed":100,"minRam":134217728,"maxRam":134217728,"arch":"x86_64","os":"CentOS
5.3 (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"vncPassword":"ec119fd35127c44e","params":{"rootDiskController":"ide","nicAdapter":"E1000"},"uuid":"1c64977b-db4f-415e-98ce-83b53a612493","disks":[{"id":14,"name":"ROOT-9","mountPoint":"/export/home/chandan/asf4-131-170/primary","path":"ROOT-9-14-000001","size":2147483648,"type":"ROOT","storagePoolType":"NetworkFilesystem","storagePoolUuid":"f42a60c2-0940-305b-bb14-23d8b40f42bc","deviceId":0},{"id":15,"name":"DATA-9","mountPoint":"/export/home/chandan/asf4-131-170/primary","path":"c7adbf5e2509484d8ac4959ae53e6146-000001","size":5368709120,"type":"DATADISK","storagePoolType":"NetworkFilesystem","storagePoolUuid":"f42a60c2-0940-305b-bb14-23d8b40f42bc","deviceId":1},{"id":16,"name":"NEWDISK-3","mountPoint":"/export/home/chandan/asf4-131-170/primary","path":"fba51dce35d24c19a224090f3cb597e8-000001","size":5368709120,"type":"DATADISK","storagePoolType":"NetworkFilesystem","storagePoolUuid":"f42a60c2-0940-305b-bb14-23d8b40f42bc","deviceId":2},{"id":9,"name":"CentOS
5.3(64-bit) no GUI (vSphere)","size":0,"type":"ISO","storagePoolType":"ISO","deviceId":3}],"nics":[{"deviceId":0,"networkRateMbps":495,"defaultNic":true,"uuid":"06e76915-e16d-4f60-a311-3a5fbdd7d5ab","ip":"10.1.1.66","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:48:7a:00:07","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://2254","isolationUri":"vlan://2254","isSecurityGroupEnabled":false}]},"result":false,"details":"StartCommand
failed due to Exception: java.lang.RuntimeException\nMessage: File []/vmfs/volumes/23dd55b9-15861742/ROOT-9-14-000001.vmdk
was not found\n","wait":0}}] }
> 2013-03-14 16:06:16,068 DEBUG [agent.transport.Request] (Job-Executor-87:job-93) Seq
1-970065218: Received:  { Ans: , MgmtId: 7200344900649, via: 1, Ver: v1, Flags: 110, { StartAnswer
} }
> 2013-03-14 16:06:16,070 DEBUG [agent.manager.AgentAttache] (DirectAgent-242:null) Seq
1-970065218: No more commands found
> 2013-03-14 16:06:16,073 INFO  [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-87:job-93)
Unable to start VM on Host[-1-Routing] due to StartCommand failed due to Exception: java.lang.RuntimeException
> Message: File []/vmfs/volumes/23dd55b9-15861742/ROOT-9-14-000001.vmdk was not found
> 2013-03-14 16:06:16,079 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-87:job-93)
Cleaning up resources for the vm VM[User|boron-VM-6] in Starting state
> 2013-03-14 16:06:16,081 DEBUG [agent.transport.Request] (Job-Executor-87:job-93) Seq
1-970065219: Sending  { Cmd , MgmtId: 7200344900649, via: 1, Ver: v1, Flags: 100111, [{"StopCommand":{"isProxy":false,"vmName":"i-3-9-VMSNAP","wait":0}}]
}
> 2013-03-14 16:06:16,081 DEBUG [agent.transport.Request] (Job-Executor-87:job-93) Seq
1-970065219: Executing:  { Cmd , MgmtId: 7200344900649, via: 1, Ver: v1, Flags: 100111, [{"StopCommand":{"isProxy":false,"vmName":"i-3-9-VMSNAP","wait":0}}]
}
> **On the primary Storage, the ROOT Volume is indeed missing**
> [root@nfs2 primary]# ls
> 08c2826505b646b3bf3d3e03dfc2bce2-000001-delta.vmdk  fba51dce35d24c19a224090f3cb597e8.vmdk
 ROOT-11-18.vmdk              ROOT-7-11.vmdk
> 08c2826505b646b3bf3d3e03dfc2bce2-000001.vmdk        i-3-3-VMSNAP                    
      ROOT-1-1-delta.vmdk          ROOT-8-12-000001-delta.vmdk
> 08c2826505b646b3bf3d3e03dfc2bce2-flat.vmdk          i-3-5-VMSNAP                    
      ROOT-1-1.vmdk                ROOT-8-12-000001.vmdk
> 08c2826505b646b3bf3d3e03dfc2bce2.vmdk               i-3-6-VMSNAP                    
      ROOT-2-2-delta.vmdk          ROOT-8-12-delta.vmdk
> 8c26613e6e5f344b928e0530ec4817c3                    i-3-7-VMSNAP                    
      ROOT-2-2.vmdk                ROOT-8-12.vmdk
> c7adbf5e2509484d8ac4959ae53e6146-000002-delta.vmdk  i-3-8-VMSNAP                    
      ROOT-3-3-delta.vmdk          ROOT-9-14-000002-delta.vmdk
> c7adbf5e2509484d8ac4959ae53e6146-000002.vmdk        i-3-9-VMSNAP                    
      ROOT-4-5-delta.vmdk          ROOT-9-14-000002.vmdk
> c7adbf5e2509484d8ac4959ae53e6146-flat.vmdk          i-6-10-VMSNAP                   
      ROOT-4-5.vmdk                ROOT-9-14-delta.vmdk
> c7adbf5e2509484d8ac4959ae53e6146.vmdk               r-11-VMSNAP                     
      ROOT-5-6-delta.vmdk          ROOT-9-14.vmdk
> cf04b6e2026f3369b893bdab955c3ff0                    r-4-VMSNAP                      
      ROOT-6-7-delta.vmdk          s-1-VMSNAP
> fba51dce35d24c19a224090f3cb597e8-000002-delta.vmdk  ROOT-10-17-delta.vmdk           
      ROOT-7-11-000001-delta.vmdk  v-2-VMSNAP
> fba51dce35d24c19a224090f3cb597e8-000002.vmdk        ROOT-10-17.vmdk                 
      ROOT-7-11-000001.vmdk
> fba51dce35d24c19a224090f3cb597e8-flat.vmdk          ROOT-11-18-delta.vmdk           
      ROOT-7-11-delta.vmdk
> [root@nfs2 primary]#
> ==========
> Bug Impact:
> ==========
> RevertToVMSnapshot Operations are blocked.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message