cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Funs Kessen <f...@barred.org>
Subject Re: Oracle VM (OVM) Server support
Date Fri, 25 Apr 2014 13:02:42 GMT
Hi Donal,

Thanks for your thoughts, I've replied inline.

On Thu, Apr 24, 2014 at 09:32:36PM +0000, Donal Lafferty wrote:
> Start with something stable, yet recent, e.g. 4.3  and not Master.  A lot of developers
use existing tools such as DevCloud or their local test bed.  As a consequence, Master is
not used nearly often as you'd expect.  For instance, in the summer of 2013, I would see Master
not run, not build according to instructions, and sometimes not build at all.  Work with Master
if you want to do QA or you understand CloudStack/Maven/Java and our pom.xml enough to make
fixes.  However, you want something recent to catch subtle changes.  For instance, JSON instruction
serialisation changed midway through 4.2.  If your tests targeted 4.1, they have been broken
by the end of 4.2.
>
Sounds sensible! I'll stay on 4.4 for now and will try to follow that first.
I did however get frightened a bit by the speed things move at, so was thinking
it would be good to keep up a bit and make sure that the code would work with
4.5.

> Start with the most common hardware architecture for CloudStack developers.  Otherwise
no one can duplicate the problem you are seeing.  Also, you really need a DevCloud for your
development to scale well.  I haven't seen a lot of SPARC laptops, so I'd guess you will want
to pursue x86 :)
> 
x86 is indeed the prime objective, sparc is a nice to have for now. I was
curious if there was an interest, Paul has had the question, but as Kris points
out, first x86 ;).

> WRT system VMs, it sounds like you can run the existing Xen system VMs as is provided
the template is converted to RAW.  Since you're using NFS, you'll have to seed the system
templates, which suggests that the conversion is in the bash script that sets up the system
VMs.  Is that correct?
> 
It's not in the bash script now, but I can put it in fairly straightforward, for
now I have a raw image just like there is a vpc, qcow2 etc image. But I could 
indeed just convert it on the fly.

> WRT supported environment, I suggest simply trying to start a VM on a basic network.
 To start that VM you will have implemented support discovery (handshaking required to create
an agent that will communicate with the hypervisor), storage (primary storage setup / secondary
storage setup boils down to a mount command), and interface with the OVM API.  The network
should be free, since basic networking will make use of your start VM instructions.  I would
guess the hard bit is interfacing with the OVM API, since it might use a different schema
for VM configuration and startup.  
>
This has all been done in combination with the OVM API. The configuration and
starting of the VM in combination with primary and secondary storage mounting,
copying of template to template, and template to primary. Basic networking works
and was pretty straightforward to implement. The communication with the WRT 
system VM, that is not done by means of it's own agent, is done over SSH for 
now.
Atm I'm preparing to get isolated/guest networking in there for advanced 
networks, which is one we rely on quite a bit.

> WRT agent architecture, since OVM's agent talks XML-RPC, you should try a direct connect
agent.  Direct connection agent handle remote communications.  Where the remote API cannot
handle CloudStack commands, they include a translation layer to convert between CloudStack's
RPC and the remote API.  In contrast, a "connected agent" is a bare proxy for the CloudStack
Agent, which uses the CloudStack message bus for communications.   For example, the Hyper-V
plugin implements a direct connect agent, which allows it to send JSON-RPC via HTTP requests.
 The XenServer plugin is also a direct connect agent, but it also includes a data translation
layer to convert from CloudStack commands to XAPI.  I suggest looking at the Hyper-V agent
to understand the instructions you have to implement, because they are all in one place. 
In contrast, XenServer agent's implementation is implemented in five or so classes that inherit
from each other.  Where you interact with the OVM API, it might be useful to see what the
XenServer instruction is doing.  They're trying to control the same hypervisor, no?
> 
I've gone down the route of using a direct connect agent, which conceptually ends
up translating the XML-RPC calls back and forth.
At the moment I'm borrowing from several sources at the same time, and learning
how things hang together by means of doing so. The Hypervisor and Dom0 are
vastly different beasts. Yes the Hypervisor is Xen which is the same (although
minor versions differ) there are indeed parts I took from the Xenserver code
base, but as you mentioned before Xenserver is using XAPI, while OVM is based 
on XM, with Oracle's own agent souce sitting on top of it. Effectively making
it a bit of a different beast with regard to control of pools, clusters, storage,
vifs, nics and other things.

> But... can you control OVM with libvirt?  If so, could you simply reuse the KVM agent
instead of writing your own?  It's a lot easier to reuse than write from scratch :)
>
I'm sure you could to a certain failry limited extent, but with some bits
really missing as they are not in the dom0 provided for OVM. Which you could
in theory push in if you really wanted to.
A reason I didn't go down that route is that I don't want to modify dom0, in the
end to avoid a support discussion if it would come to that and be able to say
that it's a completely stock OVM that is just centrally managed. Hence an as 
native as possible implementation.

Cheers,

Funs

> 
> > -----Original Message-----
> > From: Paul Angus [mailto:paul.angus@shapeblue.com]
> > Sent: 23 April 2014 19:08
> > To: dev@cloudstack.apache.org
> > Cc: funs@barred.org
> > Subject: RE: Oracle VM (OVM) Server support
> > 
> > So the million euro question is... which CloudStack release are you aiming
> > for?
> > 
> > The 100,000 euro question is... are you looking at x86 and/or SPARC
> > architectures?
> > 
> > Regards
> > 
> > Paul Angus
> > Cloud Architect
> > S: +44 20 3603 0540 | M: +447711418784 | T: CloudyAngus
> > paul.angus@shapeblue.com
> > 
> > -----Original Message-----
> > From: Funs Kessen [mailto:]
> > Sent: 23 April 2014 14:00
> > To: dev@cloudstack.apache.org
> > Cc: Donal Lafferty
> > Subject: Re: Oracle VM (OVM) Server support
> > 
> > Hi Donal and others,
> > 
> > I've been working part timeon the code for literally a bit over a month, based
> > on snippets I had. The need for the OVM3 integration was initially something
> > that had quite some pressure behind it when the discussion started, but
> > unfortunately evaporated.
> > The initial work I did started on the shoulders of the old Ovm plugin, but
> > changed partially, so the outline of the framework used there is still visible in
> > the ResourceBase, Discoverer, Helper, Guru and Fencer although modified.
> > The rest is not quite the same. I do have to say that some *cough*
> > refactoring *cough* is in order.
> > 
> > On Tue, Apr 22, 2014 at 09:24:02PM +0000, Donal Lafferty wrote:
> > > I took this approach with the Hyper-V plugin.  Based on my experience, I
> > think the following advice is of great value.  Could you please pass it on to
> > Funs.
> > >
> > Thanks for reaching out, it was the thing I needed to push me over the edge
> > and get it out there :)
> > 
> > > First, take Rajesh up on his request for information, because he can help
> > with the systemVM.  Without a system VM, you can only run a QuickCloud
> > deployment, which lacks networking and secondary storage flexibility.  Also,
> > rolling a system VM requires quite different expertise than operating the
> > hypervisor itself.  I don't think it's useful to learn both when there are a few
> > system VM experts already kicking around.
> > >
> > The Console Proxy, SSVM and RouterVM all work and run once the template
> > has been converted to RAW, which is the only format Ovm3 supports. RAW is
> > only supported due to lack of the tapdisk kernel module, the tools are there
> > but the module is not.
> > Under the hood Ovm3 is xen with a python agent combined with xm and a
> > couple of shared object libs. The agent is conceptually a pluggable
> > framework.
> > 
> > > Be clear on how you'll support the console VM.  It would be preferable to
> > give someone a chance to amend an existing console VM than have to write
> > one from scratch.
> > >
> > The console VM works :).
> > 
> > > Make it clear what instructions you'll support.  A base plugin need only
> > create/start/stop/delete, but maybe your users are expecting a richer
> > features set.  There seem to be four or so potential users on this mailing list.
> > Why not ask them what they need.
> > >
> > That is a good point indeed, for now I've kept our environement partially in
> > the back of my mind.
> > * NFS for Primary and Secondary Storage
> > * VLANs for VIFs on bridges (no OpenVswitch yet)
> > * Native Pooling and loose CS clusters (native clusters are on the list)
> > * VM migrations
> > The README.md on the github page, https://github.com/snuf/ovm3, has
> > more detail. I'm very open to suggestions on what should be in there, some
> > things are however "complicated" due to the limitations of what is bundeled
> > with dom0 and is natively supported by Oracle VM Manager. Some form of
> > keeping in line with regard to not modifying dom0 would be required for the
> > use case I have in mind.
> > 
> > > Get a sanity check on the agent architecture you're going to use.  Will we
> > have to install a remote agent?  Will it speak CloudStack message bus or take
> > HTTP requests?  It's worth checking with the community that there won't be
> > any breaking changes to this architecture when it comes time to integrate.
> > The same applies to storage.  In fact, the evolving storage model was an issue
> > faced by the Hyper-V plugin's storage architecture.
> > >
> > A long time ago I spoke to a couple of people on the mailinglist and my idea
> > was to do an as native as possible implementation, meaning that I was going
> > to levarage the Ovm3 agent, which talks XML-RPC, as if Oracle VM Manager
> > was talking to it. The plugin uses XML-RPC as much as possible, but for some
> > of the missing bits now ssh is used, which should switch to either a shadow-
> > agent, or augmenting the native Ovm3 agent.
> > In the future the native agent might be enriched so I do feel much for
> > keeping it as native as possible, also with respect to the initial use case.
> > 
> > > Finally, crowd source your test cases.  Obviously, you want to have a test
> > case for every instruction you're implemented and some functional tests that
> > cover a command sequence typical of a GUI operation.  What's more
> > interesting is whether you can collaborate with existing testers to speed up
> > development time and increase reliability.  In fact, I've some nice examples
> > I'd like to pass on.
> > >
> > At the moment I have a build street that rebuilds either the module or CS
> > completely depending, and can blow away the existing cloud, clean the
> > hypervisors and roll out the entire thing from scratch. It will wait till the
> > systemVMs are deployed and deploy three VMs on it.
> > A sheer lack of unit tests and just a few sanity checks are in but it needs a
> > bucket load more. The GUI has been tested a couple of times before I
> > decided to integrate the bits needed into the API.
> > 
> > > To summarise, try to collaborate.  There's a lot of expertise beyond
> > operating the hypervisor that you can safely crowd source.
> > >
> > That is good advice, somehow I find it hard to put the code out there as I feel
> > I've not given the code the attention it deserves.
> > 
> > The code is on github: https://github.com/snuf/ovm3 for those interested.
> > 
> > Cheers,
> > 
> > Funs
> > 
> > > > -----Original Message-----
> > > > From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> > > > Sent: 22 April 2014 20:45
> > > > To: dev
> > > > Cc: fkessen@schubergphilis.com
> > > > Subject: Re: Oracle VM (OVM) Server support
> > > >
> > > > He is implementing an ovm3 plugin + core patch. no support for the old
> > ovm.
> > > > When He is done he'll publish and I'll be happy to integrate and
> > > > steal the
> > > > credits;)
> > > >
> > > > On Tue, Apr 22, 2014 at 7:25 PM, Rajesh Battala
> > > > <rajesh.battala@citrix.com>
> > > > wrote:
> > > > > Nice..
> > > > > Can you share more details about the support of OVM?
> > > > >
> > > > > Thanks
> > > > > Rajesh Battala
> > > > >
> > > > > -----Original Message-----
> > > > > From: Daan Hoogland [mailto:daan.hoogland@gmail.com]
> > > > > Sent: Tuesday, April 22, 2014 9:52 PM
> > > > > To: dev
> > > > > Subject: Re: Oracle VM (OVM) Server support
> > > > >
> > > > > A schuberg philis colleague is working on it. Funs.
> > > > >
> > > > > mobile bilingual spell checker used Op 22 apr. 2014 18:18 schreef
> > > > > "Chip Childers" <chipchilders@apache.org>:
> > > > >
> > > > >> On Tue, Apr 22, 2014 at 03:38:51PM +0000, Paul Angus wrote:
> > > > >> > About a year ago there was some discussion around updating
the
> > > > >> > Oracle VM
> > > > >> Server support to the then current version.  The topic stopped
> > > > >> and I can't find anything in Jira for it.
> > > > >> >
> > > > >> > Is the support still being worked on?
> > > > >> >
> > > > >> > Regards
> > > > >> >
> > > > >> > Paul Angus
> > > > >> > Senior Consultant / Cloud Architect
> > > > >>
> > > > >> Not AFAIK
> > > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Daan
> > Need Enterprise Grade Support for Apache CloudStack?
> > Our CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-
> > infrastructure-support/> offers the best 24/7 SLA for CloudStack
> > Environments.
> > 
> > Apache CloudStack Bootcamp training courses
> > 
> > **NEW!** CloudStack 4.2.1 training<http://shapeblue.com/cloudstack-
> > training/>
> > 28th-29th May 2014, Bangalore.
> > Classromm<http://shapeblue.com/cloudstack-training/>
> > 16th-20th June 2014, Region A. Instructor led, On-
> > line<http://shapeblue.com/cloudstack-training/>
> > 23rd-27th June 2014, Region B. Instructor led, On-
> > line<http://shapeblue.com/cloudstack-training/>
> > 15th-20th September 2014, Region A. Instructor led, On-
> > line<http://shapeblue.com/cloudstack-training/>
> > 22nd-27th September 2014, Region B. Instructor led, On-
> > line<http://shapeblue.com/cloudstack-training/>
> > 1st-6th December 2014, Region A. Instructor led, On-
> > line<http://shapeblue.com/cloudstack-training/>
> > 8th-12th December 2014, Region B. Instructor led, On-
> > line<http://shapeblue.com/cloudstack-training/>
> > 
> > This email and any attachments to it may be confidential and are intended
> > solely for the use of the individual to whom it is addressed. Any views or
> > opinions expressed are solely those of the author and do not necessarily
> > represent those of Shape Blue Ltd or related companies. If you are not the
> > intended recipient of this email, you must neither take any action based
> > upon its contents, nor copy or show it to anyone. Please contact the sender if
> > you believe you have received this email in error. Shape Blue Ltd is a
> > company incorporated in England & Wales. ShapeBlue Services India LLP is a
> > company incorporated in India and is operated under license from Shape
> > Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in
> > Brasil and is operated under license from Shape Blue Ltd. ShapeBlue is a
> > registered trademark.

Mime
View raw message