incubator-jena-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paolo Castagna <>
Subject Re: Jena build (some thoughts)
Date Sun, 13 Nov 2011 02:23:26 GMT
Hi Dave

Dave Reynolds wrote:
> Interesting. I'd like to hear your impressions of that. I've been
> wondering how much real momentum and investment there is around OSGi.

The fact that an entire track on Friday at ApacheCon was all dedicated
to OSGi is a signal, these were the talks:

  - OSGi for mere mortals, Bertrand Delacretaz
  - Apache Celix - Universal OSGi?, Alexander Broekhuis
  - Deployment With Apache Karaf and ACE, Jean-Baptiste Onofré
  - Provisioning distributed OSGi applications in a cloud, Guillaume Nodet
  - Using OSGi to Build Better Software: Lessons from a Telemedicine
    Software for Smartphones and Desktop Systems, Doreen Seider

Of these, I attended only the talk on Apache Karaf.
With 5 tracks at the same time the chance of finding two or more
interesting talks at the same time is high.

However, I have been talking with developers using Apache projects for
OSGi and working for a couple of the ApacheCon sponsors which are using
OSGi heavily with all of their customers.

My overall impression is that OSGi usage is expanding and it's not
anymore limited to big projects such as Eclipse and/or J2EE containers
such as Apache Geronimo.

It requires changing the way you work, but it certainly a big advantage
to be able to fix a small bug in a bundle and push it live without
restarting servers as suppose to: remove a server from the load balancing
pool, shut the server down, deploy the new version, restart, add the
server back to the load balancing pool. All this can be automated with
tools such as Puppet, Chef or similar projects. However, is not as flexible
and tuned towards Java as OSGi is. This is what it seems to me one of the
biggest advantages which OSGi provides.

>> (I'll share more on ApacheCon when I am back).
>> Are you still using OSGi?
> Yes. Though I expect it to be just one tool we use for some components
> rather than the fundamental architecture we use all the time. 
>> Can I ask you which OSGi "runtime"(?) do you use?
>>   -
>>   -
>>   -
>>   - other?
> Primarily equinox. Felix is the other easy one to work with given the
> toolchain I use, indeed we mostly use the Felix component bundles.
> However, we also use jRuby with its OSGi integration and I have only got
> that to run reliably under equinox. 

The demo I see about Karaf what quite effective and Karaf works both with
Felix and Equinox:

  "Apache Karaf is a small OSGi based runtime which provides a lightweight
   container onto which various components and applications can be deployed.
   [...] Supports the latest OSGi 4.2 containers: Apache Felix Framework 3.0
   and Eclipse Equinox 3.6" --

See also:


>> You strongly advocated for the one jar (=bundle) approach and provided
>> strong evidence for this. I am convinced by your arguments.
> How important this is depends on the tool chain. There's increased
> support for using repositories, including Maven repositories for dynamic
> bundle resolution so the benefits of one bundle may be reducing. But for
> min config, get something work out of the box quickly, I still feel it
> has benefits.
>> However, I wonder if others might benefit from having OSGi bundles for
>> the single Jena components (or just Jena without the dependencies which
>> do not come from Jena, probably this is better...?). I also found this:
>> Have you been using it?
> No, though there are several bundle repositories around.
>> I still need to get my head around all these projects and how they
>> related each others and how they might relate to Apache Jena in the
>> context of OSGi.
> Sure.
>> Another fact is that two other projects in the incubator which are
>> currently using Apache Jena (directly or indirectly) are Clerezza and
>> Stanbol. They might benefit from us providing Jena OSGi bundles directly.
>> I need to look at the details and what they currently do (and how they
>> use Jena), but it might sense to deal with OSGi ourselves directly.
> I wanted control over generating bundles to match Jena snapshots so
> doing our own internal releases made most sense.

Yep, it makes sense.

The reason why I mentioned Clerezza and Stanbol was different though.
They both use Jena directly (as Clerezza) or indirectly (as Stanbol,
via Crelezza). I did not mentioned them to say: "you/we should use
them when we need Jena as OSGi bundle", but to say: "if they both use
OSGi, it would make sense for us to provide them the Jena bundles
directly and to all others who might want/need to use Jena in an OSGi

>> Finally, (this is me being curious) it would be interesting to know what
>> is your main driver in relation to OSGi. What's the feature/capability
>> which is a big attractor for you?
> Modularity plus dynamism.

I get the dynamism.

But, I am not sure I understand the modularity advantage... but it's
because I know very little about OSGi.

What I do not understand in relation to modularity is what OSGi gives
more in comparison to have modules for your projects (as we do for Jena)
for example. Aren't your OSGi bundles the same as the Jena modules?

> I wanted to be able to package modular components I was working on as
> some sort of plugin and be able to upgrade plugins with minimum hassle.
> The idea was to have a general platform which had standard APIs for
> things like RDF data repositories, data conversion services, web
> presentation services and API-over-RDF services and then be able to
> freely plugin new implementations of any of these with the same ease
> with which you can extend or upgrade eclipse. OSGi is a great platform
> for that. 
> What's more OSGi allows you to do this dynamically. So for example, with
> the base platform running if you decide you want to provision a new
> store (DataRepository) its just a matter of dropping a config file into
> the right directory and it is detected, read, provisioned and event
> notifications are sent with a minimum of code support needed and no
> restart. If there's a defect in one your components you edit it, save
> and the updated component autodeploys to the running platform in real
> time - no need to restart servers (well mostly).
> It hasn't all been plain sailing though - the problems have been
> tooling, startup sequencing and mindset.

Tooling? This is me again not knowing enough about OSGi and tools for
Eclipse... but the question which came into my mind is: "what kind of
capabilities/features do you need at development time when you package
your jar as an OSGi bundle as supposed to as a normal jar?

> For development we used bndtools in Eclipse. On its own that was
> fantastic, such an easy tool with very quick development cycles compared
> to the world of Maven and WTP.

With Maven one thing that gets in the way with multiple modules is the
fact that if you do a refactoring in one module you will not see changes
in the other modules using that module unless you properly configure a
dependency between Eclipse projects (which we, unfortunately, do not do
for Jena). Do you think we should/could change that?

Now the workflow is: change module A, mvn install, mvn compile for module
B. Going via the local repository just to see the consequences on dependent
modules of a refactoring of code in another module is a pain.

> The trouble is that it wasn't very Maven
> compatible and as Jena and our other projects have got sucked ever
> deeper into the Maven black hole I ended up having to use bndtools and
> Maven in combination. We managed the make that work (e.g. Brian
> developed a maven plugin for deploying bundles to a local bndtools
> repository) but it isn't very satisfying and requires use of m2e. This
> may improve, the bndtools folks had an intensive get together a few
> weeks ago and Maven compatibility was very high on their agenda but the
> new version hasn't been released yet.

Again, I do not have experience with what you describe. Are there things
we can/should change in the pom.xml files to make things easier?

> The startup issues are my fault rather than OSGi's but it is part of the
> OSGi challenge to handle the high levels of concurrency. When you start
> up a runtime you have limited control over what order bundles become
> available. Some of the facilities you use for configuring and
> provisioning services (config admin, Felix file install) are themselves
> services that are dynamically starting up. The once file install is up
> it starts trying to install the things it finds, often at the same time
> as the other services are still trying to start up. While the runtimes
> give you a basic notion of runlevel we've had some problems guaranteeing
> clean startup, particularly when it comes to embedded scripting like
> jRuby. While you are debugging in Eclipse on a decent machine all seems
> fine, but shift to a slow machine and it's easy to find there's a subtle
> race condition you hadn't thought of.
> The "mindset" issue is a subtle one. Thinking in terms of OSGi services
> and modules encourages you to think about components running in the same
> JVM and having rich interfaces. There are distributed OSGi solutions
> (Apache CXF, Paremus) but they feel very RPC like. Whereas for our work
> I'd rather be thinking in terms of web style REST APIs, with built in
> assumptions of distribution for each API. Having a nice pluggable
> framework makes it easy to think of each service you add as just another
> plug-in and only later discover it would have been better thought of as
> a distributable web services. 
>> Conclusion, next week (when I am back) I want to help Andy with the
>> testing of what we have for the first Apache Jena release. Fix problems
>> we might (or might not!) find. After we have rolled out a couple of
>> releases and we things/processes are set, I want to come back to this
>> and OSGi and look at how best we can make Apache Jena available to those
>> who might use it or are already using it via OSGi bundles. This in
>> practice might turn up something very easy to do (probably via a small
>> additional Maven plug-in).
> The existing Maven plug-in for OSGi is very capable.

This one: ?

> Dave

View raw message