mesos-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Azul (JIRA)" <>
Subject [jira] [Commented] (MESOS-2717) Qemu/KVM containerizer
Date Mon, 11 Apr 2016 10:46:25 GMT


Azul commented on MESOS-2717:

I have been following this JIRA, as I have a usage case where I could have used the KVM containerizer.

I use Mesos for CI through Jenkins and the Jenkins Mesos plugin, I had a couple of particular
1) I needed to be able to test a vagrant box, for that I needed not just virtualbox but I
needed to execute the 'vagrant up' command. This restricted me to able to use a virtualized
platforms that supported nested virtualization and virtualbox (probably only ESX would apply
here), or go down the bare-metal path and use something like mesos and the jenkins mesos plugin
for this task.
2) I need to be able to spin up full fat VMs of different types and OSs to run CI tests on
them, these are scrapped after every build.

For point 2, I could actually use Openstack or any other virtualization platform, but I don't
want to pay the maintenance burden of having an openstack installation, or pay for a vmware
There are a few 'workarounds' one can do with the default containerizer in mesos, for example:
execute a 'qemu-kvm' command and spin up an instance in a jenkins mesos build and then use
that instance for the test. It will get scrapped if the job dies.
Another option is to use something like rancher-vm, within docker with access to /dev/kvm
to allow full KVM hosts.

My perfect CI UX would be able to spin up a KVM instance, run a shell script and bring it
down again. I can already do this in Mesos with docker but I need a full fat vm

Another building block I see and could use,  would be that using marathon and a KVM containerizer
I could then build Stacks, a set of VMs where I could run whatever I choose in them (including

> Qemu/KVM containerizer
> ----------------------
>                 Key: MESOS-2717
>                 URL:
>             Project: Mesos
>          Issue Type: Wish
>          Components: containerization
>            Reporter: Pierre-Yves Ritschard
>            Assignee: Abhishek Dasgupta
> I think it would make sense for Mesos to have the ability to treat hypervisors as containerizers
and the most sensible one to start with would probably be Qemu/KVM.
> There are a few workloads that can require full-fledged VMs (the most obvious one being
Windows workloads).
> The containerization code is well decoupled and seems simple enough, I can definitely
take a shot at it. VMs do bring some questions with them here is my take on them:
> 1. Routing, network strategy
> ======================
> The simplest approach here might very well be to go for bridged networks
> and leave the setup and inter slave routing up to the administrator
> 2. IP Address assignment
> ====================
> At first, it can be up to the Frameworks to deal with IP assignment.
> The simplest way to address this could be to have an executor running
> on slaves providing the qemu/kvm containerizer which would instrument a DHCP server and
collect IP + Mac address resources from slaves. While it may be up to the frameworks to provide
this, an example should most likely be provided.
> 3. VM Templates
> ==============
> VM templates should probably leverage the fetcher and could thus be copied locally or
fetch from HTTP(s) / HDFS.
> 4. Resource limiting
> ================
> Mapping resouce constraints to the qemu command line is probably the easiest part, Additional
command line should also be fetchable. For Unix VMs, the sandbox could show the output of
the serial console
> 5. Libvirt / plain Qemu
> =================
> I tend to favor limiting the amount of necessary hoops to jump through and would thus
investigate working directly with Qemu, maintaining an open connection to the monitor to assert

This message was sent by Atlassian JIRA

View raw message