hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Savage <david.sav...@paremus.com>
Subject Re: Custom Class Loader for Hadoop M/R jobs?
Date Fri, 16 Apr 2010 09:08:48 GMT
On Fri, Apr 16, 2010 at 10:05 AM, David Savage <david.savage@paremus.com> wrote:
> Hi there,
>
> Many thanks to Caspar for the kind words. Yep we have had Hadoop
> running in an OSGi environment in the past - though it was for a proof
> of concept so I've not tracked the current Hadoop trunk to see if that
> patch still applies. I'm more than happy to try to offer advice if
> this is something you wish to pursue, though I'd also highly recommend
> the Felix mailing lists (user@felix.apache.org or
> dev@felix.apache.org) as excellent sources of OSGi wisdom.
>
> Regarding licensing again IANAL but here's the list of projects at
> apache are using OSGi (that I am aware of) so I'm pretty certain it's
> compatible...
>
> http://felix.apache.org
> http://aries.apache.org

Woops:

http://incubator.apache.org/aries/

> http://sling.apache.org
> http://tuscany.apache.org
> http://servicemix.apache.org
> http://activemq.apache.org
> http://cxf.apache.org
>
> Also I know there's been work to make other library projects such as
> Apache Commons OSGi compatible.
>
> Regarding custom classloaders I'd tend to advise checking if OSGi
> solves your problems first before trying to role your own - the
> benefits of using a common classloading model is that your components
> can then interact with other frameworks - also if you run into complex
> problems their are a wealth of experts out there who can help you -
> where as with a custom approach you are on your own.
>
> Regards,
>
> Dave
>
>> On Thu, 15 Apr 2010 10:47, Caspar MacRae wrote:
>>
>> ServiceMix and ActiveMQ are also using OSGi in their core AFAIK.
>>
>> The way this (these) problems have been solved is more of an afterthought,
>> and regardless is quite simplistic (thinking of classloader isolation in web
>> containers, app servers, and the monolithic WARs produced as a result),
>> however OSGi's approach to modularity provides a truly elegant solution.
>>
>> wrt to the ticket https://issues.apache.org/jira/browse/MAPREDUCE-243, Dave
>> Savage is a nice chap who works for Paremus (an enterprise OSGi provider) I
>> spoke with him at the recent OSGi devcon in London and he did quite a bit of
>> work to get this working for private clients - I'm sure with his expertise
>> it would not be a huge effort to get this rolling again.
>>
>> > On 15 April 2010 18:23, Segel, Mike <mse...@navteq.com> wrote:
>> >
>> > Ok, so are you using OSGi's code, or just writing your own code to
>> > implement a published API/specification?
>> >
>> > I realize this is getting off topic, so you may just want to e-mail me
>> > directly.
>> >
>> > Thx
>> >
>> > -Mike
>> >
>> > -----Original Message-----
>> > From: Marcel May [mailto:marcel....@consol.de]
>> > Sent: Thursday, April 15, 2010 11:35 AM
>> > To: general@hadoop.apache.org
>> > Subject: Re: Custom Class Loader for Hadoop M/R jobs?
>> >
>> >
>> > On Apr 15, 2010, at 6:27 PM, Segel, Mike wrote:
>> >
>> > > Caspar,
>> > >
>> > > While IANAL, I did a first blush read of OSGi's Specification License,
>> > Version 1.0. on the website link:
>> > http://www.osgi.org/Specifications/Licensing as well as the license on the
>> > download page: http://www.osgi.org/Download/Release4V42?info=nothanks.
>> > >
>> > > In a nutshell, OSGi's Specification License, Version 1.0. is incompatible
>> > with Apache's.
>> > >
>> > > That alone can explain why Apache projects won't touch OSGi.
>> > >
>> >
>> > http://felix.apache.org/
>> >
>> > > From a more practical issue, this problem is solved in other Apache
>> > projects, so why not borrow from another Apache project like Tomcat?
>> > >
>> > > Does that make sense? It appears OSGi is a non-starter.
>> > >
>> > > HTH
>> > >
>> > > -Mike
>> > ...
>> >
>> > Cheers,
>> > Marcel
>> >
>> >
>> > The information contained in this communication may be CONFIDENTIAL and is
>> > intended only for the use of the recipient(s) named above.  If you are not
>> > the intended recipient, you are hereby notified that any dissemination,
>> > distribution, or copying of this communication, or any of its contents, is
>> > strictly prohibited.  If you have received this communication in error,
>> > please notify the sender and delete/destroy the original message and any
>> > copy of it from your computer or paper files.
>> >
>

Mime
View raw message