incubator-cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcus Sorensen <shadow...@gmail.com>
Subject Re: QEMU support in CloudStack
Date Fri, 08 Feb 2013 19:07:01 GMT
I should also mention that in such a dev environment I would be
concerned that the code changes required to run might accidentally get
merged! Or at least screw you up if you switch branches.

On Fri, Feb 8, 2013 at 12:03 PM, Marcus Sorensen <shadowsor@gmail.com> wrote:
> Well, they're changes to the cloudstack code itself. It's sort of
> tough to say "here's devcloud-kvm, which works fine if you run it
> under Linux or VMware Fusion, but if you apply these patches to the
> code you pull down, you can run it in VirtualBox".  I don't mind
> sharing how it's done, but I don't think it belongs in a how-to. We
> should probably just decide if we want to make it work in the code.
> It's probably not of much use for dev anyway unless we go through a
> bit more testing and ensure that there aren't any edge cases that
> would cause someone to pull their hair out.
>
> On Fri, Feb 8, 2013 at 11:59 AM, Sebastien Goasguen <runseb@gmail.com> wrote:
>>
>> On Feb 8, 2013, at 7:57 PM, Marcus Sorensen <shadowsor@gmail.com> wrote:
>>
>>> Hmm, looks like /usr/sbin/virt-what in the system vms is returning
>>> 'qemu', rather than 'kvm', so cloud-early-config fails to do any
>>> setup. Forcing virt-what to return kvm in place of qemu gets things
>>> going, but perhaps we'd change cloud-early-config to do the same
>>> things for kvm|qemu.
>>>
>>> So with those three changes, we have an environment that generally
>>> seems to work and can be QA'd.
>>
>>
>> Could you post those changes to the devcloud-kvm page ?
>>
>> I know someone who is trying to use devcloud-kvm using qemu within devcloud.
>>
>> thanks,
>>
>> -sebastien
>>
>>>
>>> On Fri, Feb 8, 2013 at 11:42 AM, Marcus Sorensen <shadowsor@gmail.com>
wrote:
>>>> Looks like it would work, but the centos 6.3 libvirt isn't new enough.
>>>> libvirt says 'unknown os type hvm' on centos, even though I've
>>>> verified that os type of 'hvm' works qemu and without kvm modules on
>>>> ubuntu 12.04.
>>>>
>>>> I upgraded libvirt to a fedora version, and it worked (at least the
>>>> system vms started coming up, need to wait and see if functions work).
>>>> Changes made:
>>>>
>>>> IsHVMEnabled returns true always
>>>> hardcoded _hypervisorType to 'qemu' rather than 'kvm'
>>>>
>>>> Obviously this was just for the test, we would make these changes some
>>>> other way.
>>>>
>>>> On Fri, Feb 8, 2013 at 11:15 AM, Edison Su <Edison.su@citrix.com> wrote:
>>>>> Wondering, how do you get devcloud-kvm work?
>>>>
>>>> by modprobing kvm, kvm_intel in the guest it can run virtual machines.
>>>> Host system needs to have nested=1 set in it's kvm_intel or kvm_amd
>>>> kernel module parameters (default on many distributions now).
>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>>>>> Sent: Friday, February 08, 2013 10:02 AM
>>>>>> To: Alex Huang
>>>>>> Cc: Sebastien Goasguen; Wido den Hollander; cloudstack-
>>>>>> dev@incubator.apache.org
>>>>>> Subject: Re: QEMU support in CloudStack
>>>>>>
>>>>>> I'm running a quick sanity test here... just seeing if switching
out kvm with
>>>>>> qemu works, all else the same. Looks like there's a setHvsType for
the
>>>>>> LibvirtVMDef as well, that's hardcoded to 'kvm'.
>>>>>> That should be easy to adjust as well, assuming everything just runs
with
>>>>>> these changes.
>>>>>>
>>>>>> On Fri, Feb 8, 2013 at 10:54 AM, Alex Huang <Alex.Huang@citrix.com>
wrote:
>>>>>>> In that case, why not create two resources with the kvm resource
>>>>>> extending the qemu resource and do what Marcus suggests here in the
>>>>>> qemu resource?
>>>>>>>
>>>>>>> Effectively then we have an agent for qemu and one for kvm and
they each
>>>>>> can carry their own capabilities.
>>>>>>>
>>>>>>> --Alex
>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Marcus Sorensen [mailto:shadowsor@gmail.com]
>>>>>>>> Sent: Friday, February 08, 2013 9:41 AM
>>>>>>>> To: Sebastien Goasguen
>>>>>>>> Cc: Wido den Hollander; cloudstack-dev@incubator.apache.org
>>>>>>>> Subject: Re: QEMU support in CloudStack
>>>>>>>>
>>>>>>>> You would in theory have to disable the check in the agent
startup
>>>>>>>> code that looks for the kvm kernel modules, and then libvirt
should
>>>>>>>> just fall back to qemu for everything automatically.
>>>>>>>>
>>>>>>>> In LibvirtComputingResource.java, comment out the check for
>>>>>>>> IsHVMEnabled, and then rmmod any kvm modules, then try to
do stuff
>>>>>>>> with that version of cloudstack.
>>>>>>>>
>>>>>>>> On Fri, Feb 8, 2013 at 10:10 AM, Sebastien Goasguen
>>>>>>>> <runseb@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> On Feb 8, 2013, at 3:07 PM, Wido den Hollander <wido@widodh.nl>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> On 02/08/2013 10:34 AM, Dave Cahill wrote:
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> Recently I encountered two "nested virtualization"
use cases
>>>>>>>>>>> which
>>>>>>>> made me
>>>>>>>>>>> want QEMU hypervisor support in CloudStack. I'm
interested to
>>>>>>>>>>> hear if anyone else is interested in this feature,
and any notes
>>>>>>>>>>> on how it should be implemented.
>>>>>>>>>>>
>>>>>>>>>>> Here is a good explanation from OpenStack docs
[2] on why they
>>>>>>>> support QEMU:
>>>>>>>>>>> "From the perspective of the Compute service,
the QEMU hypervisor
>>>>>>>>>>> is
>>>>>>>> very
>>>>>>>>>>> similar to the KVM hypervisor. Both are controlled
through
>>>>>>>>>>> libvirt, both support the same feature set, and
all virtual
>>>>>>>>>>> machine images that are compatible with KVM are
also compatible
>>>>>>>>>>> with QEMU. The main
>>>>>>>> difference is
>>>>>>>>>>> that QEMU does not support native virtualization.
Consequently,
>>>>>>>>>>> QEMU
>>>>>>>> has
>>>>>>>>>>> worse performance than KVM and is a poor choice
for a production
>>>>>>>>>>> deployment."
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> So, I've been reading into the code and found this
on my Ubuntu
>>>>>> systems.
>>>>>>>>>>
>>>>>>>>>> root@stack01:~# ls -l /usr/bin/kvm lrwxrwxrwx 1 root
root 18 Oct
>>>>>>>>>> 4 02:44 /usr/bin/kvm -> qemu-system-
>>>>>>>> x86_64
>>>>>>>>>> root@stack01:~#
>>>>>>>>>>
>>>>>>>>>> Imho Qemu is Qemu and KVM only comes into play when
the kernel
>>>>>>>> module 'kvm' and 'kvm_amd' or 'kvm_intel' is loaded.
>>>>>>>>>>
>>>>>>>>>>> Here are the use cases I encountered:
>>>>>>>>>>>
>>>>>>>>>>> [Use case: Dev environment]
>>>>>>>>>>>    Wanted to use Vagrant [1] to create a portable
multi-node dev
>>>>>>>>>>> environment; however Vagrant uses VirtualBox,
which doesn't
>>>>>>>>>>> support
>>>>>>>> KVM.
>>>>>>>>>>>    Also, devcloud uses VirtualBox and devcloud-kvm
uses
>>>>>>>>>>> kvm-within-
>>>>>>>> kvm. I
>>>>>>>>>>> imagine maintenance of devcloud and devcloud-kvm
would be easier
>>>>>>>>>>> if devcloud-kvm could use VirtualBox too.
>>>>>>>>>>>    Note: Of course, I'm aware of devcloud-kvm
as an alternative
>>>>>>>>>>> for this use case, and I'll be looking into that
next.
>>>>>>>>>>>
>>>>>>>>>>> [Use case: Demo environment]
>>>>>>>>>>>    We may want to spin up a multi-node CloudStack
install in
>>>>>>>>>>> Amazon
>>>>>>>> AWS
>>>>>>>>>>> for demo purposes.
>>>>>>>>>>>    Again, AWS instances don't support KVM, so
this is not
>>>>>>>>>>> possible
>>>>>>>> without
>>>>>>>>>>> QEMU support.
>>>>>>>>>>>
>>>>>>>>>>> [Implementation ideas]
>>>>>>>>>>>    The management server currently does a check
for KVM support
>>>>>>>> ("kvm-ok")
>>>>>>>>>>> on the host, and refuses to add the host if that
fails. I think
>>>>>>>>>>> this check could be removed, as the agent setup
scripts will fail
>>>>>>>>>>> anyway if the user is trying to setup a certain
hypervisor on a
>>>>>>>>>>> machine which doesn't
>>>>>>>> support
>>>>>>>>>>> it.
>>>>>>>>>>
>>>>>>>>>> This way you could do nested virtualization indeed,
but it could
>>>>>>>>>> also hurt
>>>>>>>> users who have their BIOS set to disabled and could lead
to long
>>>>>> debugging.
>>>>>>>>>>
>>>>>>>>>>>    Create a new setting in agent.properties like
"use_qemu",
>>>>>>>>>>> with a default of "false". If the person deploying
CloudStack
>>>>>>>>>>> agent sets this to "true", cloud-setup-agent
and other setup
>>>>>>>>>>> scripts would ignore lack of
>>>>>>>> KVM
>>>>>>>>>>> support as long as QEMU support was available.
>>>>>>>>>>
>>>>>>>>>> cloud-setup-agent generates a agent.properties, so
at that point
>>>>>>>>>> it
>>>>>>>> doesn't know that the user intents to use the system without
KVM
>>>>>> support.
>>>>>>>>>>
>>>>>>>>>>>    Lastly, when creating the libvirt XML file
for a VM, set
>>>>>>>>>>> hypervisor to QEMU rather than KVM in the XML
file depending on
>>>>>> the config setting.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> That's not hard coded. The Agent does a getCapabilities()
call to
>>>>>>>>>> libvirt
>>>>>>>> which returns a list of possible emulators.
>>>>>>>>>>
>>>>>>>>>> /usr/bin/kvm is just one of them which is returned
and matches the
>>>>>>>> architecture.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Wido,
>>>>>>>>>
>>>>>>>>> So can I add a "KVM" host that would in fact just use
qemu ?
>>>>>>>>> How would I do that ?
>>>>>>>>>
>>>>>>>>> -sebastien
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Wido
>>>>>>>>>>
>>>>>>>>>>> Thanks for reading,
>>>>>>>>>>> Dave.
>>>>>>>>>>>
>>>>>>>>>>> [1] http://www.vagrantup.com/
>>>>>>>>>>> [2]
>>>>>>>>>>> http://docs.openstack.org/trunk/openstack-
>>>>>>>> compute/install/yum/content/qemu.html
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>

Mime
View raw message