cloudstack-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrei Mikhailovsky <and...@arhont.com>
Subject Re: Fail to add RBD ceph as Primary storage on Cloudstack 4.1+Centos6.4 KVM host
Date Tue, 20 Aug 2013 08:38:33 GMT


Well, I do not think that it is down to 2.6.32 kernel that is causing your issue as CS and
RBD integration doesn't use kernel module. Please correct me if I am wrong, but I think it
uses userspace libraries. Try asking people on OFTC irc #ceph, there should be people using
centos/RH. 

When I tried rbd with cs using centos 6.4 I have compiled the latest libvirt and qemu from
sources and didn't have that many issues apart from heavy load server reboots. I was running
fio with 4 files each threading 16 random reads/writes with block size of 4K. Larger block
sizes didn't give me any issues. At that time I had to use centos because of the infiniband
support issues which i had with ubuntu, but the latest ofed drivers sorted out the issue and
I am happily back to ubuntu. 

Do you need to use Centos for a reason? Taking into account that Ceph is heavily developed
and tested using Ubuntu servers, I would use it instead of Centos, unless there are issues
with hardware, etc. which do not work in Ubuntu. 

Andrei 
----- Original Message -----

From: "不坏阿峰" <onlydebian@gmail.com> 
To: users@cloudstack.apache.org 
Sent: Tuesday, 20 August, 2013 3:18:36 AM 
Subject: Re: Fail to add RBD ceph as Primary storage on Cloudstack 4.1+Centos6.4 KVM host


thank you for your response. 

i have done Widoh's way too and running well on ubuntu. so i know it 
is the centos some kernel and features too old. 
Hope have some one make a success deploy on Centos 6.4 

2013/8/20 Andrei Mikhailovsky <andrei@arhont.com>: 
> I've been testing centos 6.4 extensively with rbd and had a lot of stability issues where
the host server would freeze or reboot during high load benchmarks, especially with small
block sizes. I have switched to ubuntu 12.04 with 3.5 kernel and didn't see this behaviour.
Just my two cents. 
> 
> P.S. i've not had any issues with adding rbd cluster to cloudstack following the Widoh's
howto on the net. 
> 
> Andrei 
> ----- Original Message ----- 
> 
> From: "不坏阿峰" <onlydebian@gmail.com> 
> To: users@cloudstack.apache.org 
> Sent: Monday, 19 August, 2013 1:40:46 PM 
> Subject: Fail to add RBD ceph as Primary storage on Cloudstack 4.1+Centos6.4 KVM host

> 
> Environment: 
> ################## 
> Cloudstack 4.1.1, Ceph 0.6.1.7, Qemu 0.12.1.2-2.355 with RBD enable 
> 
> I have added ceph RBD as primary successfully with ubuntu 12.04 kvm. 
> but fail on centos 6.4 kvm host in cloudstack. 
> 
> i did some test on centos kvm host, i think can use rbd to access ceph 
> (have the ceph.conf on KVM host) 
> ####################### 
> [root@centos-kvm01 ~]# qemu-img -v | grep rbd 
> Supported formats: raw cow qcow vdi vmdk cloop dmg bochs vpc vvfat 
> qcow2 qed parallels nbd blkdebug host_cdrom host_floppy host_device 
> file rbd 
> 
> [root@centos-kvm01 ~]# rbd -m 192.168.250.15 -p volumes ls 
> 08b376f0-5af4-4b46-b41e-d83dc0f93a70 
> 1e9f8d76-94d9-4d56-a8a6-fdff32ba9d1b 
> b0f7fae3-716b-4210-a89d-af63c6e30859 
> 
> [root@centos-kvm01 ~]# rados lspools 
> data 
> metadata 
> rbd 
> volumes 
> cloudstack 
> 
> [root@centos-kvm01 ~]# rbd create test --size 4096 
> [root@centos-kvm01 ~]# rbd ls 
> test 
> 
> cloudstack log: 
> #################### 
> 2013-08-19 17:28:25,683 DEBUG [agent.transport.Request] 
> (AgentManager-Handler-9:null) Seq 26-368640010: Processing: { Ans: , 
> MgmtId: 345050143793, via: 26, Ver: v1, Flags: 10, 
> [{"Answer":{"result":false,"details":"java.lang.NullPointerException\n\tat 
> com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.createStoragePool(LibvirtStorageAdaptor.java:540)\n\tat

> com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:111)\n\tat

> com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:104)\n\tat

> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2304)\n\tat

> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1094)\n\tat

> com.cloud.agent.Agent.processRequest(Agent.java:525)\n\tat 
> com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)\n\tat 
> com.cloud.utils.nio.Task.run(Task.java:83)\n\tat 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)\n\tat

> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\n\tat

> java.lang.Thread.run(Thread.java:679)\n","wait":0}}] } 
> 2013-08-19 17:28:25,683 DEBUG [agent.transport.Request] 
> (catalina-exec-1:null) Seq 26-368640010: Received: { Ans: , MgmtId: 
> 345050143793, via: 26, Ver: v1, Flags: 10, { Answer } } 
> 2013-08-19 17:28:25,683 DEBUG [agent.manager.AgentManagerImpl] 
> (catalina-exec-1:null) Details from executing class 
> com.cloud.agent.api.ModifyStoragePoolCommand: 
> java.lang.NullPointerException 
> at com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.createStoragePool(LibvirtStorageAdaptor.java:540)

> at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:111)

> at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:104)

> at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2304)

> at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1094)

> at com.cloud.agent.Agent.processRequest(Agent.java:525) 
> at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852) 
> at com.cloud.utils.nio.Task.run(Task.java:83) 
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) 
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
> at java.lang.Thread.run(Thread.java:679) 
> 
> 2013-08-19 17:28:25,689 WARN [cloud.storage.StorageManagerImpl] 
> (catalina-exec-1:null) Unable to establish a connection between 
> Host[-26-Routing] and Pool[207|RBD] 
> com.cloud.exception.StorageUnavailableException: Resource 
> [StoragePool:207] is unreachable: Unable establish connection from 
> storage head to storage pool 207 due to java.lang.NullPointerException 
> at com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.createStoragePool(LibvirtStorageAdaptor.java:540)

> at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:111)

> at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:104)

> at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2304)

> at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1094)

> at com.cloud.agent.Agent.processRequest(Agent.java:525) 
> at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852) 
> at com.cloud.utils.nio.Task.run(Task.java:83) 
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) 
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
> at java.lang.Thread.run(Thread.java:679) 
> at com.cloud.storage.StorageManagerImpl.connectHostToSharedPool(StorageManagerImpl.java:1637)

> at com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:1402) 
> at com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:147) 
> at org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd.execute(CreateStoragePoolCmd.java:123)

> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162) 
> at com.cloud.api.ApiServer.queueCommand(ApiServer.java:505) 
> at com.cloud.api.ApiServer.handleRequest(ApiServer.java:355) 
> at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:302) 
> at com.cloud.api.ApiServlet.doGet(ApiServlet.java:66) 
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617) 
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) 
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)

> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)

> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)

> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)

> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) 
> at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) 
> at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:615) 
> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)

> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293) 
> at org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889) 
> at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:744)

> at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2282)

> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) 
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
> at java.lang.Thread.run(Thread.java:679) 
> 2013-08-19 17:28:25,690 WARN [cloud.storage.StorageManagerImpl] 
> (catalina-exec-1:null) No host can access storage pool Pool[207|RBD] 
> on cluster 3 
> 2013-08-19 17:28:25,693 INFO [cloud.api.ApiServer] 
> (catalina-exec-1:null) Failed to add storage pool 
> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message