hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Savage <david.sav...@paremus.com>
Subject Re: Custom Class Loader for Hadoop M/R jobs?
Date Fri, 16 Apr 2010 09:05:05 GMT
Hi there,

Many thanks to Caspar for the kind words. Yep we have had Hadoop
running in an OSGi environment in the past - though it was for a proof
of concept so I've not tracked the current Hadoop trunk to see if that
patch still applies. I'm more than happy to try to offer advice if
this is something you wish to pursue, though I'd also highly recommend
the Felix mailing lists (user@felix.apache.org or
dev@felix.apache.org) as excellent sources of OSGi wisdom.

Regarding licensing again IANAL but here's the list of projects at
apache are using OSGi (that I am aware of) so I'm pretty certain it's
compatible...

http://felix.apache.org
http://aries.apache.org
http://sling.apache.org
http://tuscany.apache.org
http://servicemix.apache.org
http://activemq.apache.org
http://cxf.apache.org

Also I know there's been work to make other library projects such as
Apache Commons OSGi compatible.

Regarding custom classloaders I'd tend to advise checking if OSGi
solves your problems first before trying to role your own - the
benefits of using a common classloading model is that your components
can then interact with other frameworks - also if you run into complex
problems their are a wealth of experts out there who can help you -
where as with a custom approach you are on your own.

Regards,

Dave

> On Thu, 15 Apr 2010 10:47, Caspar MacRae wrote:
>
> ServiceMix and ActiveMQ are also using OSGi in their core AFAIK.
>
> The way this (these) problems have been solved is more of an afterthought,
> and regardless is quite simplistic (thinking of classloader isolation in web
> containers, app servers, and the monolithic WARs produced as a result),
> however OSGi's approach to modularity provides a truly elegant solution.
>
> wrt to the ticket https://issues.apache.org/jira/browse/MAPREDUCE-243, Dave
> Savage is a nice chap who works for Paremus (an enterprise OSGi provider) I
> spoke with him at the recent OSGi devcon in London and he did quite a bit of
> work to get this working for private clients - I'm sure with his expertise
> it would not be a huge effort to get this rolling again.
>
> > On 15 April 2010 18:23, Segel, Mike <mse...@navteq.com> wrote:
> >
> > Ok, so are you using OSGi's code, or just writing your own code to
> > implement a published API/specification?
> >
> > I realize this is getting off topic, so you may just want to e-mail me
> > directly.
> >
> > Thx
> >
> > -Mike
> >
> > -----Original Message-----
> > From: Marcel May [mailto:marcel....@consol.de]
> > Sent: Thursday, April 15, 2010 11:35 AM
> > To: general@hadoop.apache.org
> > Subject: Re: Custom Class Loader for Hadoop M/R jobs?
> >
> >
> > On Apr 15, 2010, at 6:27 PM, Segel, Mike wrote:
> >
> > > Caspar,
> > >
> > > While IANAL, I did a first blush read of OSGi's Specification License,
> > Version 1.0. on the website link:
> > http://www.osgi.org/Specifications/Licensing as well as the license on the
> > download page: http://www.osgi.org/Download/Release4V42?info=nothanks.
> > >
> > > In a nutshell, OSGi's Specification License, Version 1.0. is incompatible
> > with Apache's.
> > >
> > > That alone can explain why Apache projects won't touch OSGi.
> > >
> >
> > http://felix.apache.org/
> >
> > > From a more practical issue, this problem is solved in other Apache
> > projects, so why not borrow from another Apache project like Tomcat?
> > >
> > > Does that make sense? It appears OSGi is a non-starter.
> > >
> > > HTH
> > >
> > > -Mike
> > ...
> >
> > Cheers,
> > Marcel
> >
> >
> > The information contained in this communication may be CONFIDENTIAL and is
> > intended only for the use of the recipient(s) named above.  If you are not
> > the intended recipient, you are hereby notified that any dissemination,
> > distribution, or copying of this communication, or any of its contents, is
> > strictly prohibited.  If you have received this communication in error,
> > please notify the sender and delete/destroy the original message and any
> > copy of it from your computer or paper files.
> >

Mime
View raw message