incubator-jena-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dave Reynolds <>
Subject Re: Jena build (some thoughts)
Date Sat, 12 Nov 2011 16:12:03 GMT
Hi Paolo,

On Sat, 2011-11-12 at 07:00 -0800, Paolo Castagna wrote: 
> Dave Reynolds wrote:
> > I'd like (as in, I would put work into this) to also have osgi support. 
> > 
> > Suggest:
> > 
> > o the jena-one-jar would also be marked as an OSGI bundle (easy to do
> > via maven bundle plugin)
> > 
> > o a third top level download, jena-complete, which is the jena-one-jar
> > plus dependent jars (i.e. xerces, slf4j/log4j etc) packaged as an single
> > jar/OSGI bundle. 
> > 
> > The latter is not *necessary* for OSGi support but the way we/I
> > currently work with OSGi makes it easier to have reasonably chunky
> > self-contained bundles than do the fine grain dependency management at
> > the OSGi level. 
> > 
> > This can also be used via "java -jar jena-complete.jar ..." to run any
> > of the command line utilities which would save some support list load on
> > classpath advice :)
> Hi Dave,
> one "side effect" that ApacheCon had on me is a strong exposure to OSGi
> and contacts (from experts on this) which could translate in help or
> guidance.

Interesting. I'd like to hear your impressions of that. I've been
wondering how much real momentum and investment there is around OSGi.

> (I'll share more on ApacheCon when I am back).
> Are you still using OSGi?

Yes. Though I expect it to be just one tool we use for some components
rather than the fundamental architecture we use all the time. 

> Can I ask you which OSGi "runtime"(?) do you use?
>   -
>   -
>   -
>   - other?

Primarily equinox. Felix is the other easy one to work with given the
toolchain I use, indeed we mostly use the Felix component bundles.
However, we also use jRuby with its OSGi integration and I have only got
that to run reliably under equinox. 

> You strongly advocated for the one jar (=bundle) approach and provided
> strong evidence for this. I am convinced by your arguments.

How important this is depends on the tool chain. There's increased
support for using repositories, including Maven repositories for dynamic
bundle resolution so the benefits of one bundle may be reducing. But for
min config, get something work out of the box quickly, I still feel it
has benefits.

> However, I wonder if others might benefit from having OSGi bundles for
> the single Jena components (or just Jena without the dependencies which
> do not come from Jena, probably this is better...?). I also found this:
> Have you been using it?

No, though there are several bundle repositories around.

> I still need to get my head around all these projects and how they
> related each others and how they might relate to Apache Jena in the
> context of OSGi.


> Another fact is that two other projects in the incubator which are
> currently using Apache Jena (directly or indirectly) are Clerezza and
> Stanbol. They might benefit from us providing Jena OSGi bundles directly.
> I need to look at the details and what they currently do (and how they
> use Jena), but it might sense to deal with OSGi ourselves directly.

I wanted control over generating bundles to match Jena snapshots so
doing our own internal releases made most sense.

> Finally, (this is me being curious) it would be interesting to know what
> is your main driver in relation to OSGi. What's the feature/capability
> which is a big attractor for you?

Modularity plus dynamism.

I wanted to be able to package modular components I was working on as
some sort of plugin and be able to upgrade plugins with minimum hassle.
The idea was to have a general platform which had standard APIs for
things like RDF data repositories, data conversion services, web
presentation services and API-over-RDF services and then be able to
freely plugin new implementations of any of these with the same ease
with which you can extend or upgrade eclipse. OSGi is a great platform
for that. 

What's more OSGi allows you to do this dynamically. So for example, with
the base platform running if you decide you want to provision a new
store (DataRepository) its just a matter of dropping a config file into
the right directory and it is detected, read, provisioned and event
notifications are sent with a minimum of code support needed and no
restart. If there's a defect in one your components you edit it, save
and the updated component autodeploys to the running platform in real
time - no need to restart servers (well mostly).

It hasn't all been plain sailing though - the problems have been
tooling, startup sequencing and mindset.

For development we used bndtools in Eclipse. On its own that was
fantastic, such an easy tool with very quick development cycles compared
to the world of Maven and WTP. The trouble is that it wasn't very Maven
compatible and as Jena and our other projects have got sucked ever
deeper into the Maven black hole I ended up having to use bndtools and
Maven in combination. We managed the make that work (e.g. Brian
developed a maven plugin for deploying bundles to a local bndtools
repository) but it isn't very satisfying and requires use of m2e. This
may improve, the bndtools folks had an intensive get together a few
weeks ago and Maven compatibility was very high on their agenda but the
new version hasn't been released yet.

The startup issues are my fault rather than OSGi's but it is part of the
OSGi challenge to handle the high levels of concurrency. When you start
up a runtime you have limited control over what order bundles become
available. Some of the facilities you use for configuring and
provisioning services (config admin, Felix file install) are themselves
services that are dynamically starting up. The once file install is up
it starts trying to install the things it finds, often at the same time
as the other services are still trying to start up. While the runtimes
give you a basic notion of runlevel we've had some problems guaranteeing
clean startup, particularly when it comes to embedded scripting like
jRuby. While you are debugging in Eclipse on a decent machine all seems
fine, but shift to a slow machine and it's easy to find there's a subtle
race condition you hadn't thought of.

The "mindset" issue is a subtle one. Thinking in terms of OSGi services
and modules encourages you to think about components running in the same
JVM and having rich interfaces. There are distributed OSGi solutions
(Apache CXF, Paremus) but they feel very RPC like. Whereas for our work
I'd rather be thinking in terms of web style REST APIs, with built in
assumptions of distribution for each API. Having a nice pluggable
framework makes it easy to think of each service you add as just another
plug-in and only later discover it would have been better thought of as
a distributable web services. 

> Conclusion, next week (when I am back) I want to help Andy with the
> testing of what we have for the first Apache Jena release. Fix problems
> we might (or might not!) find. After we have rolled out a couple of
> releases and we things/processes are set, I want to come back to this
> and OSGi and look at how best we can make Apache Jena available to those
> who might use it or are already using it via OSGi bundles. This in
> practice might turn up something very easy to do (probably via a small
> additional Maven plug-in).

The existing Maven plug-in for OSGi is very capable.


View raw message