cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcel Offermans <marcel.offerm...@luminis.nl>
Subject Re: Proposal to use OSGi for component modularity
Date Wed, 30 May 2012 21:54:22 GMT
Alex, one of the mentors, pointed me to this thread. I've been doing OSGi projects for over
10 years now and am involved in Felix, ACE and Celix (all OSGi related projects here at Apache).

> From: Alex Huang <Alex.Huang@citrix.com>
> 
> CloudStack is built as a product with modularity enforced by interfaces.  It had its
own component management via ComponentLocator.  While that framework made it easy for a small
group of developers to work together, it has outlived its usefulness.  Now that we are expecting
more developers to join Apache CloudStack, we're looking for the next component framework
 to scale code development.  This framework needs to manage the following:
> 	- life cycles of the components
> 	- enforce boundaries between components
> 	- easily allow components to be upgraded/downgraded
> 	- database upgrades for the component

Except for the last one, these requirements can definitely be fulfilled by the OSGi framework
"out of the box".

> The most successful of such an application is Eclipse, which uses framework defined by
OSGi.  We're planning to use that for our component framework.

I will probably be grilled for this one, but I think Eclipse is both the best known and worst
example of using OSGi. I would definitely caution against blindly adopting some of its extensions
to OSGi.

> I like to gather opinions about
> 
> 1. Whether to use OSGi or there should be other frameworks to consider?

Eclipse had its own framework before it moved to OSGi. At that time they also considered JMX
as that was being used by JBoss. By now, JBoss is also moving towards OSGi. There have been
many more examples of small and large applications that have either moved completely to OSGi
or used it as a way to allow "plugins". Obviously, there have also been projects that have
considered OSGi and chose not to use it.

I definitely am not getting paid to plug any books here, but I found the recently released
"Java Application Architecture" book by Kirk Knoernschild a very good introduction to modularity
in general. It mentions OSGi but focusses more on the benefits of a modular architecture and
shows how that can help you both without and with OSGi. See: http://www.kirkk.com/modularity/

> 2. If OSGi, whether to use Equinox or Apache Felix?

You should develop against the OSGi specification, so you can use any framework that implements
this spec. Probably it will be easier to use Felix because it's Apache licensed but most,
if not all projects that use OSGi make sure they run on all major open source frameworks:
Felix, Equinox and Knopflerfish.

> 3. The tool to help integrate the OSGi framework into tomcat (Sling?)

There are many components (bundles) in various projects that you can use for that. Is Tomcat
a requirement? I'm asking because OSGi defines an HttpService that is implemented using different
Java web servers (from Jetty to small embedded ones) that will allow you to use Servlets and
(with some extra bundles) JSP pages, JAX-RS based REST endpoints, etc.

> 4. Anything we can use to upgrade/downgrade database versions?

This is an interesting one.

First of all, if you have a modular application, having a single database below it can be
a bit of a deal-breaker from a modularity point of view. In a modular world, a good starting
point is to make every component (or every service) responsible for storing its own data.
That way you don't break any abstractions in one layer by exposing the underlying storage
schema below.

That being said, I know nothing about your schema, requirements, etc. so please do take this
remark above for what it's worth.

Are you looking for including schema upgrades and downgrades in the normal life cycle of bundles,
so you can upgrade or downgrade your database whenever you upgrade or downgrade a specific
bundle? Do you have one big schema, or can different components contribute new things to an
aggregated schema? Do you want to allow up- and downgrades from and to arbitrary versions?
Etc. This is definitely an area that needs to be explored and specified some more to give
good advice.


> From: Chris Custine <chris.custine@gmail.com>
> 
>> 1. Whether to use OSGi or there should be other frameworks to consider?
> Personally, I would absolutely love to see osgi explored.  We have been using osgi with
great success for ServiceMix for several years and have learned volumes about the ins and
outs.  For a controlled server environment like Cloudstack, I think it would be ideal.  There
are some learning curves, and some extra work involved, but the rewards are plenty.

Learning the APIs to use OSGi is definitely not that hard.

Designing a big application in a truly modular way is, and you definitely need to give yourself
(as a project) some room to experiment because this is very much an iterative process.

> From: Frank Zhang <Frank.Zhang@citrix.com>
> 
> I have some different thoughts here
> 
>> On Tuesday, May 29, 2012 at 10:29 PM, Alex Huang wrote:
>>> 1. Whether to use OSGi or there should be other frameworks to consider?
>> Personally, I would absolutely love to see osgi explored.  We have been using
>> osgi with great success for ServiceMix for several years and have learned
>> volumes about the ins and outs.  For a controlled server environment like
>> Cloudstack, I think it would be ideal.  There are some learning curves, and
>> some extra work involved, but the rewards are plenty.
> 
> First I don't think CloudStack needs to update or remove bundle on the fly.

Fair enough, probably not everybody needs this. This is only one of the potential benefits
of using OSGi though. Enforcing a modular design and decoupling components via services are
other benefits that, in my experience, also help you a lot during development and in the long
run maintaining code.

> This feature looks ideal, but considering a cluster management nodes, for 
> example four running mgmt nodes, how to propagate bundle upgrade on
> one mgmt node to others? And how to guarantee all bundles get working only
> after all mgmt nodes gets upgraded? I don't know if OSGI is capable of this problem.

OSGi itself defines a framework where each component or bundle has its own life cycle. It
contains a service registry that allows them to communicate without "leaking" any implementation
details. OSGi also comes with a compendium: a catalog of standardized service interfaces.

It does not automatically solve the problem of updating groups or clusters of nodes. However,
Apache ACE is a provisioning server that is specifically designed to manage and update large
numbers of nodes (called "targets"). ACE uses jclouds to work with different cloud providers
(and jclouds in fact already supports cloudstack) so maybe that is something to consider when
tackling this problem.

> To me runtime upgrade introduces much complexity, at least to CloudStack.

Again, fair enough, this is a decision you as a project will need to make.

> Second I think the bundle boundary of OSGI is too tiny. I may be wrong here, 
> as I am not OSGI expert. I was thinking it is a good idea to treat every interface as
> a service? For example, logging, I hope the logging package used by every bundles
> is linked in it,  instead of packaging it as a logging bundle. CloudStack should focus
on
> its own bundle, like network bundle, storage bundle. As CloudStack is a integration project
> it uses a lot of thirdparty libraries, spending too much time in repackaging libraries
I thought
> should be my linkage dependency to OSGI service distracts us from main business. 
> I know there are some projects repackaging common java libraries, I just simply suspect
if
> it is the best practice for plugin system.

The granularity of bundles is something you can decide for yourself obviously.

On the example of logging. OSGi actually defines a LogService as part of its compendium. If
you decide to use that, you can leave the choice of which component to use that actually implements
(and publishes) that service to the people integrating your bundles into their application.
In this specific case you probably even want to keep working if there is no LogService at
all. You definitely do not want to worry about developing your own logging bundle, and there
is absolutely no need for it either. There are many existing ones. Pick one, stick to the
common API, and your users can still choose the one they like best.

To get a feeling for the third party libraries, do you have a list somewhere? Probably some
of us here have used at least some of them in OSGi in the past and might be able to share
some advice on how to deal with them. In theory, you have the following options:

1) someone already packaged it as an OSGi bundle, potentially even exposing a service;
2) you package it as a bundle that shares its packages, so it can be used by others just like
a normal library;
3) you embed it in the bundle or bundles that use it and don't expose it (if it really is
just an implementation detail).

> And about extender. Now CloudStack itself is responsible for loading all components and
starting them.
> As I know in OSGI this is work of extender, does it mean we have to write our extender?
I know we can
> use dependency in MANIFEST file, but I don't know how it guarantees loading order. To
me using scattering files
> to describe inter-dependency to guarantee loading order makes people hard to know the
exact loading sequence.
> And what I dislike MANIFEST file most is you have to recompile every time you change
it.

OSGi uses some excellent frameworks to manage service dependencies and those are the preferred
mechanism to ensure that components start in the right order. You simply declare the dependencies
for every component (and you can do that with XML, annotations or even a fluent Java API).
The framework will then take care of these dependencies for you, and you can focus on the
implementation.

Extenders can work on top of the mechanism I described above.

Your point about having to recompile whenever you change the manifest is true. Then again,
whenever you change a class, you have to compile and regenerate the bundle as well. Luckily
there are some great development tools available that make this whole process fast and painless
(my personal preference here is to use BndTools, an Eclipse plugin, as it gives me the fastest
possible edit/save/run cycle). Ant and Maven support are also available, and widely used.

>>> 2. If OSGi, whether to use Equinox or Apache Felix?
>> 
>> I am biased having been involved with Felix and Karaf for many years, but I
>> would encourage you to look at Karaf and Felix as a base for the container
>> (eplained more below).  You will get assistance from within the Apache
>> community by way of Karaf, Felix, and several other projects that also use
>> these frameworks (Geronimo, Camel, Sling, probably many others now) and
>> those projects will mutually benefit from input and contributions via
>> Cloudstack.
> 
> Does Apache have any OSGI runtime to ease developing? I am not sure if I use
> correct notion. I mean something like Eclipse Virgo that builds on OSGI engine.

I'm not sure if I understand what you're asking, but BndTools (see http://bndtools.org/ ),
which I mentioned above, might be what you're looking for. Then again, there are others.

Greetings, Marcel


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message