Mailing-List: contact cloudstack-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: cloudstack-dev@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of Alex.Huang@citrix.com
 designates 66.165.176.89 as permitted sender)
From: Alex Huang <Alex.Huang@citrix.com>
To: "cloudstack-dev@incubator.apache.org"
	<cloudstack-dev@incubator.apache.org>
Date: Wed, 14 Nov 2012 06:45:52 -0800
Subject: RE: [DISCUSS] OSGi framework for plugins and more?
Thread-Topic: [DISCUSS] OSGi framework for plugins and more?
Thread-Index: Ac3CTECjv7PrFGuiQ4iEPgJlnzeBoQAKEoaA
Message-ID: 
 <B1DF26ECC0458748AC97CECE2DA98D41012FA1E1CAFA@SJCPMAILBOX01.citrite.net>
References: 
 <CAOvkMoZTGgYFzqifaGKPJNs=Tps=FcDz-okfvjqwrF9Tw24n5w@mail.gmail.com>
	<CCC7F0A0.1C0D0%kelven.yang@citrix.com>
 <CAAxZHddjhM=ABF8KxycSVvAexsZ4b4kkXnOzOD=tE312xthh+A@mail.gmail.com>
In-Reply-To: 
 <CAAxZHddjhM=ABF8KxycSVvAexsZ4b4kkXnOzOD=tE312xthh+A@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Here's my two-bits on OSGi.  I actually started a thread like this sometime=
 ago to which Mohammad reference.  I then did some research on what OSGi ca=
n do.  The problems I looked at using OSGi for just isn't solvable by OSGi.

For example,=20
- How to contain fault within an availability zone.
- How to do rolling upgrade and phase out the rolling upgrades over a span =
of days to deal with the time that it might take.
- How to do database upgrades/downgrades for the plugins
- How to scale different components differently

OSGi does solve some problems
- How to enable and disable plugins on a production system but I'm not quit=
e sure how reliable that is.  Even eclipse asks you to restart eclipse afte=
r adding a plugin.

I think after looking at this, then I decided that=20

- For modularity, nothing is better than compilation boundaries.  The probl=
em with some of the plugins is that it depends on cloud-core and cloud-serv=
er.  It shouldn't .  All plugins must build to cloud-api only.  Since all i=
nterfaces of CloudStack is in cloud-api (if you think about that then cloud=
-api is basically the OSGi bundle), that's sufficient to separate the plugi=
ns.
- For lifecycle of plugins, it probably requires that we switch to deploy i=
n something like Karaf before we can achieve runtime lifecycle changes.  I'=
m not sure it's entire necessary and it doesn't take care of a plugin's dat=
abase versioning problem.
- To resolve the other problems, we basically need to break cloudstack into=
 separate processes.  Hence I've proposed the idea of disaggregating clouds=
tack.

--Alex

> -----Original Message-----
> From: Charles Moulliard [mailto:ch007m@gmail.com]
> Sent: Wednesday, November 14, 2012 1:41 AM
> To: cloudstack-dev
> Subject: Re: [DISCUSS] OSGi framework for plugins and more?
>=20
> Hi Kelven,
>=20
> As Spring/Spring DM can be used on Apache Karaf (multi-container OSGI
> runtime), that should not be a problem to use what you have done on Karaf=
.
> Maybe we can talk about that next on IRC channel (@ch007M)
>=20
> Regards,
>=20
>=20
>=20
>=20
> On Tue, Nov 13, 2012 at 10:05 PM, Kelven Yang
> <kelven.yang@citrix.com>wrote:
>=20
> > The refactoring work that I did on Spring exploration is an incremental
> > approach trying to improve and encourage modularity of CloudStack, I
> hope
> > it will help CloudStack in the future to add OSGi in the future.
> >
> > As we have a huge amount of legacy code base, it will be less risky to =
do
> > incremental changes, and Spring effort here is the closest match of wha=
t
> > we have in our existing code base. What we are trying to achieve is to
> > first enable internal java modules to be more component-ized, our
> > customized component framework actually provides 99% things we need,
> > instead of us to fix the missing 1%, I feel that it might be worth to j=
ust
> > switch to a more standard framework with a minimal effort. It is  that
> > simple :-).  It is also easier to get things moving, in the end, if we
> > have a well component-ized system internally, it will be easier to adop=
t
> > any "the best" option like OSGi
> >
> > Kelven
> >
> >
> > On 11/13/12 8:09 AM, "Mohammad Nour El-Din"
> <nour.mohammad@gmail.com>
> > wrote:
> >
> > >Hi
> > >
> > >Had typos in this message the correct one is here:
> > >
> > >hi
> > >
> > >   we already had this discussion before earlier when ACS was just
> donated
> > >to ASF.
> > >
> > >I am not an OSGi expert at all :), but I would like to know what is th=
e
> > >problem that needs to be solved rather than discussing or jumping into
> the
> > >solution first
> > >
> > >OSGi has a lot of magnificent features idd but do we really need them =
all,
> > >and also what does all these features add to us in the context of the
> > >context of the cloud. to be honest I don't have a definite answer and
> > >that's why I would suggest to list what we need to achieve regarding t=
hat
> > >part and look into the alternatives and choose what is best for us, OS=
Gi
> > >is
> > >one option for sure
> > >
> > >Feedback/input ?
> > >On Tue, Nov 13, 2012 at 8:30 AM, Mohammad Nour El-Din <
> > >nour.mohammad@gmail.com> wrote:
> > >
> > >> hi
> > >>
> > >>    we already had this discussion before earlier when ACS was just
> > >>donated
> > >> to ASF.
> > >>
> > >> I am not an OSGi expert at all :), but I would like to know what is =
the
> > >> problem that needs to be solved rather than discussing the solution
> > >>first
> > >>
> > >> OSGi has a lot of magnificent features idd but do we really need the=
m
> > >>all,
> > >> and also what does all these features add to us in the context of th=
e
> > >> context of the cloud. to be honest I don have a definite answer and
> > >>that's
> > >> why I would suggest to list what we need to achieve regarding that p=
art
> > >>and
> > >> look into the alternatives and choose what is best for us, OSGi is o=
ne
> > >>of
> > >> them for sure
> > >>
> > >> Feedback/input ?
> > >>
> > >> Sent from my Samsung Galaxy S3
> > >> Apologies for any typos
> > >>
> > >> On Nov 13, 2012 7:56 AM, "Hugo Trippaers"
> > >><HTrippaers@schubergphilis.com>
> > >> wrote:
> > >> >
> > >> >
> > >> >
> > >> > > -----Original Message-----
> > >> > > From: Kelven Yang [mailto:kelven.yang@citrix.com]
> > >> > > Sent: Monday, November 12, 2012 11:31 PM
> > >> > > To: cloudstack-dev@incubator.apache.org
> > >> > > Subject: Re: [DISCUSS] OSGi framework for plugins and more?
> > >> > >
> > >> > >
> > >> > >
> > >> > > On 11/12/12 1:34 PM, "Chip Childers" <chip.childers@sungard.com>
> > >> wrote:
> > >> > >
> > >> > > >On Mon, Nov 12, 2012 at 4:12 AM, Hugo Trippaers
> > >> > > ><HTrippaers@schubergphilis.com> wrote:
> > >> > > >> Hey  all,
> > >> > > >>
> > >> > > >> With the recent discussions on refactoring CloudStack and the
> > >> working
> > >> > > >>going into javelin I would like to discuss using OSGi. The
> > >>background
> > >> > > >>is that I have been struggling with ideas on how to setup a pl=
ugin
> > >> > > >>system for CloudStack that would allow plugins to be separate
> > >> entities
> > >> > > >>which can be release independently from CloudStack core. Mainl=
y
> to
> > >> > > >>deal with the current non-asf components but for future
> expansion
> > >>as
> > >> > > well.
> > >> > > >>
> > >> > > >> While at ApacheConEU I had the chance to discuss these ideas
> with
> > >> one
> > >> > > >>of our mentors after his talk about OSGi. I'm pretty charmed b=
y
> > >>OSGi
> > >> > > >>and the options it provides. It's a well thought out system th=
at
> > >> allow
> > >> > > >>true modularity and pluggability. With the amount of support i=
n
> > >>the
> > >> > > >>java industry it's a standard that feels very mature and a saf=
e
> > >>bet,
> > >> > > >>one I would prefer to any homegrown plugin system. It supports
> > >> > > >>versioning of components, strict separation of classes between
> > >> > > >>components and all kind of funky features like hot-plugging an=
d
> > >> > > >>hot-replace. Using OSGi would mean that people can supply
> bundles
> > >> with
> > >> > > >>functionality which are maintained separately from the 'main
> code'
> > >> > > >>without having to worry about how to integrate it with the cor=
e.
> > >>Just
> > >> > > >>putting the module in the right directory should be enough to =
get
> > >> > > CloudStack to see and use the bundle.
> > >> > > >>Upgrades happen the same way, new version of an authenticator,
> > >>just
> > >> > > >>replace the bundle and let the framework replace it with havin=
g
> to
> > >> > > >>shutdown the server at all.
> > >> > > >>
> > >> > > >> As we are discussing making CloudStack more modular, I would
> > >>like to
> > >> > > >>propose to start using OSGi for this. It is a bit of a learnin=
g
> > >>curve
> > >> > > >>to start with, but one we can get help with from our mentors. =
I'm
> > >> > > >>already working on setting up a proof of concept for a plugin
> > >>system
> > >> > > >>using OSGi together with a colleague to show how it works, cod=
e
> is
> > >> > > >>always better than words.
> > >> > > >>
> > >> > > >> So what are your thoughts?
> > >> > > >>
> > >> > > >> Cheers,
> > >> > > >>
> > >> > > >> Hugo
> > >> > > >>
> > >> > > >
> > >> > > >I'm not familiar enough with OSGi to understand the tradeoffs o=
f
> > >>that
> > >> > > >vs other frameworks, but I'd suggest that Kelvin weigh in here.
> > >>The
> > >> > > >work that he's doing on the Javelin branch is similar, and ther=
e
> > >>might
> > >> > > >be an argument for Spring instead.
> > >> > > >
> > >> > > >Kelvin, I know you just responded on the other thread about the
> > >> > > >relative timing of a switch.  Want to weigh in on the OSGi
> > >>approach's
> > >> > > >technical merit vs. other options?
> > >> > >
> > >> > > It will be nice to see the OSGi technical merit vs. other option=
 in
> > >> details.
> > >> > > Laying out these basic but fundamental frames may not benefit a =
lot
> > >>in
> > >> the
> > >> > > short term, but we may get fully paid in long term. Spring can o=
nly
> > >> give us
> > >> > > solution on compile-time/load-time component integration, it
> focuses
> > >> > > more on internal component wiring, OSGi seems to focus more at
> > >> runtime, I
> > >> > > think these two may be complementary to each other.
> > >> >
> > >> > To a certain extent these technologies can be used together, but n=
ot
> > >>in
> > >> this way it seems. OSGi is a framework that focusses on separation o=
f
> > >>code
> > >> in various modules in such a way that other modules can not see and
> work
> > >> with the classes in other modules excluding via well defined service=
s.
> > >>This
> > >> is a fundamental choice that touches the core of how CloudStack woul=
d
> be
> > >> put together. Instead of a single codebase (or at runtime a single
> > >> classloader) where modules would be loaded via (for example) the
> spring
> > >> framework based on an xml definition, the core would be an empty
> > >>framework
> > >> and modules work be plugged in and provide a certain service. This c=
an
> > >>be
> > >> done at load time or runtime, that up to the implementers. For examp=
le
> > >>say
> > >> the core module would need a vm provisioned it would ask the service
> > >> registry if there was a service able to provide this functionality, =
if
> > >> there was that service would be asked to perform that task. Here is =
a
> > >>short
> > >> post that describes the context far better than I can:
> > >> http://elron.us/?p=3D95 .The real benefits are in also in the modula=
rity,
> > >> because the framework is very strict on what bundles/interfaces are
> > >>exposed
> > >> and required, using the proper interfaces and limiting yourself to
> > >>exposed
> > >> interfaces is enforced by the framework.
> > >> >
> > >> > Hugo
> > >> > >
> > >> > >
> > >> > > >
> > >> > > >-chip
> > >> >
> > >>
> > >>
> > >
> > >
> > >--
> > >Thanks
> > >- Mohammad Nour
> > >----
> > >"Life is like riding a bicycle. To keep your balance you must keep mov=
ing"
> > >- Albert Einstein
> >
> >
>=20
>=20
> --
> Charles Moulliard
> Apache Committer / Sr. Enterprise Architect (RedHat)
> Twitter : @cmoulliard | Blog : http://cmoulliard.blogspot.com