cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Milamber <milam...@apache.org>
Subject Re: Issue with CS 4.1 on CentOS/KVM with iSCSI/CLVM: lost volume group
Date Sat, 13 Apr 2013 23:37:16 GMT


Le 13/04/2013 23:29, Milamber a ecrit :
> Hello,
>
> I've a issue with a CloudStack 4.1 installation on CentOS 6.4 with the 
> hypervisor KVM.
>
> The primary storage is CLVM on iSCSI array (Dell Equallogic) on 3 
> nodes (hosts) with 1 Cloudmgr server.
>
> I've installed and configure a CLVM with locking_type = 3 and 
> wait_for_locks = 30. Cman like the slides: 
> http://fr.slideshare.net/MarcusLSorensen/cloud-stack-clvm.
> I have a volume group share over iscsi on each node.
>
> The issue come when I've 2 ou 3 nodes up. When I try to create a new 
> instance (from ISO or Template), somes times (> 70%) the creation 
> failed. (That works some times.
> (==> no problem when I only have one active node, I can create a lot 
> of VM simultaneous without issue)
>
> After some log inspection, I've found that the volume is "disappear" 
> from the node (which create the VM).
>
> In libvirtd log, we have when it's failed:
> 2013-04-13 20:41:56.812+0000: 28373: error : 
> virStorageBackendVolOpenCheckMode:1023 : cannot stat file 
> '/dev/vg_baie/23761696-cca7-4d09-a65f-fa9d88fbd987': No such file or 
> directory
> 2013-04-13 20:41:56.841+0000: 28373: error : virCommandWait:2345 : 
> internal error Child process (/sbin/vgchange -aln vg_baie) unexpected 
> exit status 5:   Can't deactivate volume group "vg_baie" with 16 open 
> logical volume(s)
>

Please note: the volume group is correctly displays with a virsh 
pool-list command before the creation VM's attempt.


> (a virsh pool-list show that the vg has disappeared, I must restart 
> cloudstack-agent for enabled it. Note: the pool isn't with the 
> autostart option)
>
> On management.log, we have:
>
> 2013-04-13 22:58:02,022 DEBUG [cloud.storage.StorageManagerImpl] 
> (Job-Executor-38:job-228) Creating volume: Vol[68|vm=68|ROOT]
> 2013-04-13 22:58:02,022 DEBUG [cloud.storage.StorageManagerImpl] 
> (Job-Executor-38:job-228) Trying to create in Pool[200|CLVM]
> 2013-04-13 22:58:02,027 DEBUG [agent.transport.Request] 
> (Job-Executor-38:job-228) Seq 1-1321467941: Sending  { Cmd , MgmtId: 
> 130177108533, via: 1, Ver: v1, Flags: 100111, 
> [{"storage.CreateCommand":{"volId":68,"pool":{"id":200,"uuid":"3425857b-3c3c-4e2a-8eba-115cefbe982b","host":"localhost","path":"/vg_baie","port":0,"type":"CLVM"},"diskCharacteristics":{"size":107374182400,"tags":[],"type":"ROOT","name":"ROOT-68","useLocalStorage":false,"recreatable":false,"diskOfferingId":5,"volumeId":68,"hyperType":"KVM"},"wait":0}}]

> }
> 2013-04-13 22:58:02,077 DEBUG [agent.transport.Request] 
> (AgentManager-Handler-2:null) Seq 1-1321467941: Processing:  { Ans: , 
> MgmtId: 130177108533, via: 1, Ver: v1, Flags: 110, 
> [{"storage.CreateAnswer":{"requestTemplateReload":false,"result":false,"details":"Exception:

> com.cloud.utils.exception.CloudRuntimeException\nMessage: 
> org.libvirt.LibvirtException: Storage pool not found: no pool with 
> matching uuid\nStack: com.cloud.utils.exception.CloudRuntimeException: 
> org.libvirt.LibvirtException: Storage pool not found: no pool with 
> matching uuid\n\tat 
> com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.getStoragePool(LibvirtStorageAdaptor.java:427)\n\tat

> com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.getStoragePool(KVMStoragePoolManager.java:71)\n\tat

> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:1272)\n\tat

> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1073)\n\tat

> com.cloud.agent.Agent.processRequest(Agent.java:525)\n\tat 
> com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)\n\tat 
> com.cloud.utils.nio.Task.run(Task.java:83)\n\tat 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)\n\tat

> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\n\tat

> java.lang.Thread.run(Thread.java:679)\n","wait":0}}] }
> 2013-04-13 22:58:02,077 DEBUG [agent.manager.AgentAttache] 
> (AgentManager-Handler-2:null) Seq 1-1321467941: No more commands found
> 2013-04-13 22:58:02,077 DEBUG [agent.transport.Request] 
> (Job-Executor-38:job-228) Seq 1-1321467941: Received:  { Ans: , 
> MgmtId: 130177108533, via: 1, Ver: v1, Flags: 110, { CreateAnswer } }
> 2013-04-13 22:58:02,077 DEBUG [cloud.storage.StorageManagerImpl] 
> (Job-Executor-38:job-228) Unable to create volume Vol[68|vm=68|ROOT]
> 2013-04-13 22:58:02,079 INFO  [cloud.vm.VirtualMachineManagerImpl] 
> (Job-Executor-38:job-228) Unable to contact resource.
> com.cloud.exception.StorageUnavailableException: Resource 
> [StoragePool:200] is unreachable: Unable to create Vol[68|vm=68|ROOT]
>         at 
> com.cloud.storage.StorageManagerImpl.prepare(StorageManagerImpl.java:3483)
>         at 
> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:748)
>         at 
> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:471)
>         at 
> org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:212)
>         at 
> org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
>         at 
> com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3860)
>         at 
> com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3458)
>         at 
> com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3444)
>         at 
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>         at 
> org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:379)
>         at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162)
>         at 
> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437)
>         at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at 
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:679)
> 2013-04-13 22:58:02,081 DEBUG [cloud.vm.VirtualMachineManagerImpl] 
> (Job-Executor-38:job-228) Cleaning up resources for the vm 
> VM[User|Centos100go] in Starting state
> 2013-04-13 22:58:02,083 DEBUG [agent.transport.Request] 
> (Job-Executor-38:job-228) Seq 1-1321467942: Sending  { Cmd , MgmtId: 
> 130177108533, via: 1, Ver: v1, Flags: 100111, 
> [{"StopCommand":{"isProxy":false,"vmName":"i-2-68-VM","wait":0}}] }
> (and after a message: 
> com.cloud.exception.InsufficientServerCapacityException: Unable to 
> create a deployment for VM[User...)
>
>
> Perhaps that is a configuration issue on CLVM ? or iSCSI attach?
>
> I'm not sure if CloudStack support the iSCSI/CLVM with KVM hypervisor? 
> yes/no.
>
> I must change for a NFS primary storage (over iSCSI) ? or a 
> SharedMountPoint (GFS?).
>
> That is very annoying for me, please have some ideas?
>
> Thanks.
>
> Milamber
>
>


Mime
View raw message