cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wido den Hollander <w...@widodh.nl>
Subject Re: Problem in adding Ceph RBD storage to CloudStack
Date Mon, 22 Jul 2013 19:02:05 GMT
Hi,

On 07/22/2013 08:55 PM, Satoshi Shimazaki wrote:
> Hi Wido,
>
> Thank you for your comment.
>
>> What I see is "No such file or directory", so that RBD image does not
> exist.
>> It seems like a copy didn't succeed but now CloudStack thinks that the
> image does exist.
>> Does "libvirt-pool" have a RBD image with the name
> 5e5d9b40-270b-44af-9479-782175556c47 ?
> No, it does not.
>
> There is no "5e5d9b40-270b-44af-9479-782175556c47" as below.
>

Oh, I meant this:

$ rbd -p libvirt-pool info 5e5d9b40-270b-44af-9479-782175556c47

> virsh # pool-list
> Name                 State      Autostart
> -----------------------------------------
> 3900a5bf-3362-392b-8bd0-57b10ef47bb5 active     no
> b39ca2cd-65ea-46d5-8a71-c3a4ef95028e active     no
> cd6520d6-bfc3-3537-9600-7f044e11ddb1 active     no
>
> virsh # vol-list 3900a5bf-3362-392b-8bd0-57b10ef47bb5
> Name                 Path
> -----------------------------------------
> 6ff9719f-3e4d-4ff5-ab67-154e30c936c2
> libvirt-pool/6ff9719f-3e4d-4ff5-ab67-154e30c936c2
> 8555f35f-3ed8-436b-895a-04e88e7327e0
> libvirt-pool/8555f35f-3ed8-436b-895a-04e88e7327e0
> cd3688ab-e37b-4866-9ea7-4051b670a323
> libvirt-pool/cd3688ab-e37b-4866-9ea7-4051b670a323
>
>
>> What I see is "No such file or directory", so that RBD image does not
> exist.
>> It seems like a copy didn't succeed but now CloudStack thinks that the
> image does exist.
> I agree, but I don't understand why it occured...
>

I can't tell now. Can you try an install from an ISO to see if that works?

And if that works, can you deploy a fresh template to see what the copy 
does?

It will run qemu-img twice:

1. Secondary Storage -> RBD
2. RBD -> RBD

In 4.2 the second step will be a RBD clone operation btw.

Wido

> - VMs(Root disk) on NFS : It works.
> - VMs(Root disk) on RBD : It doesn't work
> - Data Disk on RBD (attach to VMs on NFS) : It works.
>
> Is there any other points to be checked?
>
>
> Thanks,
> Satoshi Shimazaki
>
>
>
> 2013/7/23 Wido den Hollander <wido@widodh.nl>
>
>> Hi,
>>
>>
>> On 07/22/2013 07:56 PM, Satoshi Shimazaki wrote:
>>
>>> Hi Wido,
>>>
>>> I'm in the project with Kimi and Nakajima-san.
>>>
>>> [root@rx200s7-07m ~]# ceph -v
>>> ceph version 0.61.4 (**1669132fcfc27d0c0b5e5bb93ade59**d147e23404)
>>>
>>> Same version is installed into all the hosts (KVM host and Ceph nodes).
>>>
>>> Here is KVM Agent log.
>>> http://pastebin.com/5yG1uBuj
>>> I had set the log level "DEBUG" and failed to create 2 instances
>>> ,"RBDVM-shimazaki-1" and "RBDVM-shimazaki-2".
>>>
>>>
>> What I see is "No such file or directory", so that RBD image does not
>> exist.
>>
>> It seems like a copy didn't succeed but now CloudStack thinks that the
>> image does exist.
>>
>> Does "libvirt-pool" have a RBD image with the name 5e5d9b40-270b-44af-9479-
>> **782175556c47 ?
>>
>> Wido
>>
>>
>>   Thanks,
>>> Satoshi Shimazaki
>>>
>>>
>>>
>>>
>>> 2013/7/23 Kimihiko Kitase <Kimihiko.Kitase@citrix.co.jp>
>>>
>>>   Hi Wido
>>>>
>>>> Thanks for you comment.
>>>>
>>>> If we create vm on the NFS primary storage and mount additional disk on
>>>> the RBD storage, it works fine.
>>>> If we check vm from virt manager, there is no virtual disk. So we believe
>>>> the problem should be vm configuration...
>>>>
>>>> We will check ceph version tomorrow.
>>>>
>>>> Thanks
>>>> Kimi
>>>>
>>>> -----Original Message-----
>>>> From: Wido den Hollander [mailto:wido@widodh.nl]
>>>> Sent: Monday, July 22, 2013 11:43 PM
>>>> To: dev@cloudstack.apache.org
>>>> Subject: Re: Problem in adding Ceph RBD storage to CloudStack
>>>>
>>>> Hi,
>>>>
>>>> On 07/22/2013 02:25 PM, Kimihiko Kitase wrote:
>>>>
>>>>> Wido, Thank you very much.
>>>>>
>>>>> CloudStack: 4.1.0
>>>>> QEMU: 1.5.50
>>>>> Libvirt: 0.10.2
>>>>>
>>>>
>>>> What version of Ceph on the nodes?
>>>>
>>>> $ ceph -v
>>>>
>>>>
>>>>> We will set "DEBUG" on the agent tomorrow. But the following is command
>>>>>
>>>> CloudStack issue. We got this command at KVM host.
>>>>
>>>>>
>>>>> [root@rx200s7-07m ~]# ps -ef|grep 1517
>>>>> root     16099     1 27 19:36 ?        00:00:12 /usr/libexec/qemu-kvm
>>>>>
>>>> -name i-2-1517-VM -S -M pc-i440fx-1.6 -enable-kvm -m 256 -smp
>>>> 1,sockets=1,cores=1,threads=1 -uuid e67f1707-fe92-3426-978d-**
>>>> 0441d5000d6a
>>>> -no-user-config -nodefaults -chardev
>>>> socket,id=charmonitor,path=/**var/lib/libvirt/qemu/i-2-1517-**
>>>> VM.monitor,server,nowait
>>>> -mon chardev=charmonitor,id=**monitor,mode=control -rtc base=utc
>>>> -no-shutdown
>>>> -boot dc -drive
>>>> file=rbd:libvirt-pool/**cd3688ab-e37b-4866-9ea7-**
>>>> 4051b670a323:id=libvirt:key=**AQC7OuZReMndFxAAY/**
>>>> qUwLbvfod6EMvgVWU21g==:auth_**supported=cephx\;none:mon_**
>>>> host=192.168.10.20\:6789,if=**none,id=drive-virtio-disk0,**
>>>> format=raw,cache=none
>>>> -device
>>>> virtio-blk-pci,bus=pci.0,addr=**0x4,drive=drive-virtio-disk0,**
>>>> id=virtio-disk0
>>>> -drive
>>>> if=none,media=cdrom,id=drive-**ide0-1-0,readonly=on,format=**
>>>> raw,cache=none
>>>> -device ide-drive,bus=ide.1,unit=0,**drive=drive-ide0-1-0,id=ide0-**1-0
>>>> -netdev
>>>> tap,fd=27,id=hostnet0,vhost=**on,vhostfd=29 -device
>>>> virtio-net-pci,netdev=**hostnet0,id=net0,mac=02:00:0a:**b9:00:16
>>>>    ,bus=pci.
>>>> 0,addr=0x3 -chardev pty,id=charserial0 -device
>>>> isa-serial,chardev=**charserial0,id=serial0 -usb -device
>>>> usb-tablet,id=input0
>>>> -vnc 0.0.0.0:3 -vga cirrus -device
>>>> virtio-balloon-pci,id=**balloon0,bus=pci.0,addr=0x5
>>>>
>>>>>
>>>>>
>>>> The argument to Qemu seems just fine, so I think the problem is not in
>>>> CloudStack.
>>>>
>>>> Wido
>>>>
>>>>   Thanks
>>>>> Kimi
>>>>>
>>>>> -----Original Message-----
>>>>> From: Wido den Hollander [mailto:wido@widodh.nl]
>>>>> Sent: Monday, July 22, 2013 7:47 PM
>>>>> To: dev@cloudstack.apache.org
>>>>> Subject: Re: Problem in adding Ceph RBD storage to CloudStack
>>>>>
>>>>> Hi,
>>>>>
>>>>> On 07/22/2013 12:43 PM, Kimihiko Kitase wrote:
>>>>>
>>>>>> It seems secondary storage vm could copy template to primary storage
>>>>>>
>>>>> successfully, but created VM doesn't point this vol..
>>>>
>>>>> If we create vm manually and add this vol as boot vol, it works fine..
>>>>>>
>>>>>>
>>>>> Which version of CloudStack are you using?
>>>>>
>>>>> What is the Qemu version running on your hypervisor and what libvirt
>>>>>
>>>> version?
>>>>
>>>>>
>>>>> If you set the logging level on the Agent to "DEBUG", does it show
>>>>>
>>>> deploying the VM with the correct XML parameters?
>>>>
>>>>>
>>>>> I haven't seen the things you are reporting.
>>>>>
>>>>> Wido
>>>>>
>>>>>   So it seems cloudstack cannot configure VM correctly in ceph rbd
>>>>>>
>>>>> environment.
>>>>
>>>>>
>>>>>> Any idea?
>>>>>>
>>>>>> Thanks
>>>>>> Kimi
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Kimihiko Kitase [mailto:Kimihiko.Kitase@**citrix.co.jp<Kimihiko.Kitase@citrix.co.jp>
>>>>>> ]
>>>>>> Sent: Monday, July 22, 2013 7:11 PM
>>>>>> To: dev@cloudstack.apache.org
>>>>>> Subject: RE: Problem in adding Ceph RBD storage to CloudStack
>>>>>>
>>>>>> Hello
>>>>>>
>>>>>> I am in the project with Nakajima san.
>>>>>>
>>>>>> We succeeded to add RBD storage to primary storage.
>>>>>> But when we try to boot centos as user instance, it fail during system
>>>>>>
>>>>> logger process.
>>>>
>>>>> It works fine when we boot centos using NFS storage.
>>>>>> It works fine when we boot centos using NFS storage and add additional
>>>>>>
>>>>> disk from RBD storage.
>>>>
>>>>>
>>>>>> Do you have any idea to resolve this issue?
>>>>>>
>>>>>> Thanks
>>>>>> Kimi
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Takuma Nakajima [mailto:penguin.trance.2716@**gmail.com<penguin.trance.2716@gmail.com>
>>>>>> ]
>>>>>> Sent: Saturday, July 20, 2013 12:23 PM
>>>>>> To: dev@cloudstack.apache.org
>>>>>> Subject: Re: Problem in adding Ceph RBD storage to CloudStack
>>>>>>
>>>>>> I'm sorry but I forgot to tell you that the environment does not
have
>>>>>>
>>>>> the internet connection.
>>>>
>>>>> It is not allowed to make a direct connection to the internet because
>>>>>>
>>>>> of the security policy.
>>>>
>>>>>
>>>>>> Wido,
>>>>>>
>>>>>>> No, it works for me like a charm :)
>>>>>>>
>>>>>>> Could you set the Agent logging to DEBUG as well and show the
output
>>>>>>> of
>>>>>>>
>>>>>> that log? Maybe paste the log on pastebin.
>>>>>>
>>>>>>>
>>>>>>> I'm interested in the XMLs the Agent is feeding to libvirt when
>>>>>>> adding
>>>>>>>
>>>>>> the RBD pool.
>>>>>>
>>>>>> I thought the new libvirt overwrites the old one, but actually both
>>>>>>
>>>>> libvirt (with RBD and without RBD) were installed to the system. qemu
>>>> was
>>>> installed from the package and so it might have the dependency to the
>>>> libvirt installed from the package. After deleting the both libvirt
>>>> installed from source and package, then installed it from rpm package
>>>> with
>>>> RBD support, RBD storage was registered to the CloudStack successfully.
>>>>
>>>>>
>>>>>> David,
>>>>>>
>>>>>>> Why not 6.4?
>>>>>>>
>>>>>>
>>>>>> Because of no internet connection, packages in the local mirror
>>>>>>
>>>>> repository may be old.
>>>>
>>>>> I checked /etc/redhat-release and it showed the version is 6.3.
>>>>>>
>>>>>> In current state, although the RBD storage was installed, system
VMs
>>>>>> won't start with "Unable to get vms
>>>>>> org.libvirt.LibvirtException: Domain not found: no domain with
>>>>>> matching uuid 'xxxxxxxx-xxxx-xxxx-xxxx-**xxxxxxxxxxxx'" error like
>>>>>> http://mail-archives.apache.**org/mod_mbox/cloudstack-users/**
>>>>>> 201303.mbox<http://mail-archives.apache.org/mod_mbox/cloudstack-users/201303.mbox>
>>>>>> /
>>>>>> %**3CD2EE6B3265AD864EB3EA4F5C670D**256F3546C1@EXMBX01L-CRP-03.**
>>>>>> webmdhealth.
>>>>>> net%3E The uuid in the error message was not in the database of the
>>>>>> management server nor in ceph storage node.
>>>>>>
>>>>>> I tried removing host from the CloudStack and cleaning up the computing
>>>>>>
>>>>> node, but it cannot be added again to the CloudStack.
>>>>
>>>>> agent log says it attempted to connect to localhost:8250 though the
>>>>>>
>>>>> management server address is set to 10.40.1.190 in global settings.
>>>>
>>>>> management server log is here: http://pastebin.com/muGz73c0
>>>>>> (10.40.1.24 is the address of the computing node)
>>>>>>
>>>>>> Now the computing node is under rebuilding.
>>>>>>
>>>>>> Takuma Nakajima
>>>>>>
>>>>>> 2013/7/19 David Nalley <david@gnsa.us>:
>>>>>>
>>>>>>> On Thu, Jul 18, 2013 at 12:09 PM, Takuma Nakajima
>>>>>>> <penguin.trance.2716@gmail.com**> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I'm building a CloudStack 4.1 with Ceph RBD storage using
RHEL 6.3
>>>>>>>>
>>>>>>> recently
>>>>>>
>>>>>>> but it fails when adding RBD storage to primary storage.
>>>>>>>> Does anybody know about the problem?
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Why not 6.4?
>>>>>>>
>>>>>>
>>>>
>>>
>

Mime
View raw message