incubator-jena-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paolo Castagna <>
Subject Re: Jena build (some thoughts)
Date Sun, 13 Nov 2011 16:32:45 GMT
Hi Dave,
thanks for your answer (which I found useful).

Dave Reynolds wrote:
> Hi Paolo,
> On Sat, 2011-11-12 at 18:23 -0800, Paolo Castagna wrote:
>> My overall impression is that OSGi usage is expanding and it's not
>> anymore limited to big projects such as Eclipse and/or J2EE containers
>> such as Apache Geronimo.
> Interesting. I've certainly come across people who use it routinely on
> projects of all sizes but it wasn't clear to me it was growing rather
> than shrinking, that's good to know.
>>>> Finally, (this is me being curious) it would be interesting to know what
>>>> is your main driver in relation to OSGi. What's the feature/capability
>>>> which is a big attractor for you?
>>> Modularity plus dynamism.
>> I get the dynamism.
>> But, I am not sure I understand the modularity advantage... but it's
>> because I know very little about OSGi.
>> What I do not understand in relation to modularity is what OSGi gives
>> more in comparison to have modules for your projects (as we do for Jena)
>> for example. Aren't your OSGi bundles the same as the Jena modules?
> The modularity is at a finer grain size than a bundle. Working in OSGi
> encourages you to structure your solution as a set of separate component
> that will be externally wired together. In particular, I make heavy use
> of the Component Services Framework and the bnd annotation framework for
> that which makes it very easy to declare components and component
> dependencies (singleton/multiple, factories, static/dynamic).
> For example to add a new RDF store type in Modal (the modular framework
> I've been working on) I can just declare a new factory implementation
> and drop it in then my existing code can find and invoke the factory.
> Contrast that with ModelFactory which either needs extending when you
> add a new Model type or you have to provide alternative factory
> interfaces.

Now I understand.

> Of course, this is all perfectly possible using a dependency injection
> framework or a set of conventions on top of plain Java. There is no
> magic. It is simply a style that the platform supports well.

Yep. Indeed, we use Google Guice (via ServiceLoader [1]) for the things
you described above.


For a project such Apache Jena where the speed and/or number of new "things"
coming in is not very high these things might perceived just an unnecessary
overhead. However, now that Jena is in Apache, it would be good to focus on
extension points and plan for making as easy as possible to:

  - add new readers/writers for RDF IO and/or SPARQL result sets
  - add new storage/query layers
  - add new custom indexes and made them available to users via SPARQL
  - ...

 From my limited experience, these things are not as easy and they often
requires changes to the Jena code itself.

It would be good to be able to do this via simple drop-in extensions and
without the need to change or touch the Jena codebase. This IMHO could add
value to the project and help growing the community of contributors to the

>>> It hasn't all been plain sailing though - the problems have been
>>> tooling, startup sequencing and mindset.
>> Tooling? This is me again not knowing enough about OSGi and tools for
>> Eclipse... but the question which came into my mind is: "what kind of
>> capabilities/features do you need at development time when you package
>> your jar as an OSGi bundle as supposed to as a normal jar?
> Packaging as a bundle is trivial. The universal tool for that is bnd
> which you can drive directly, via Eclipse plugins like bndtools or via
> the Maven bundle plugin.
> For me the incremental development approach support by bndtools is by
> far the most attractive, especially with its support for multi-bundle
> projects. All of that is lost of you touch Maven with its
> one-artefact-per-project approach.


>>> For development we used bndtools in Eclipse. On its own that was
>>> fantastic, such an easy tool with very quick development cycles compared
>>> to the world of Maven and WTP.
>> With Maven one thing that gets in the way with multiple modules is the
>> fact that if you do a refactoring in one module you will not see changes
>> in the other modules using that module unless you properly configure a
>> dependency between Eclipse projects (which we, unfortunately, do not do
>> for Jena). Do you think we should/could change that?
> No.


> Eclipse project dependencies are the most fragile part of m2e in my
> experience, the advice I've had has been to keep that switched off and
> that certainly seems to help with some of the Maven instabilities.


>> Now the workflow is: change module A, mvn install, mvn compile for module
>> B. Going via the local repository just to see the consequences on dependent
>> modules of a refactoring of code in another module is a pain.
> Quite. In bndtools without Maven all that is dynamic, the cross project
> dependencies just worked. Lovely. In bndtools *with* Maven then you end
> up having to make sure the modules you depend on are visible in
> bndtools' idea of a repository as well so you are back to the piecemeal
> workflow you describe but with more opportunities for things to go
> wrong.

Things are clearly not perfect and different people have different preferences,
opinions, working styles, experience different pain points etc. There are also
different communities to whom we feed our software modules, etc.

You cannot imagine how much I would love to make everybody happy on this and
remove all the pain points. But, unfortunately, not always I know how to do
it or sometimes it is not even possible considering the status we are in with
the tools we have available.

Having said that, not everything is bad... if I look back a few years, I would
say that the situation is certainly improved and many of the things who used
to be done manually are now automated and we do not even think to them anymore
(after some initial effort). From my point of view the cost of maintaining
modules has been reduced as well as the unnecessary diversity.

Next week, I am going to focus on what we have and testing things for the
first Apache Release. After we have done a couple of releases in Apache,
we should come back to OSGi and explore what we can do to make life of
Apache Jena and OSGi users easier and as close as possible to "pain free".

OSGi is another important channel (as well as the Maven repository) to better
connect with advanced Apache Jena users (and therefore we should IMHO take
care of that).

>>> The trouble is that it wasn't very Maven
>>> compatible and as Jena and our other projects have got sucked ever
>>> deeper into the Maven black hole I ended up having to use bndtools and
>>> Maven in combination. We managed the make that work (e.g. Brian
>>> developed a maven plugin for deploying bundles to a local bndtools
>>> repository) but it isn't very satisfying and requires use of m2e. This
>>> may improve, the bndtools folks had an intensive get together a few
>>> weeks ago and Maven compatibility was very high on their agenda but the
>>> new version hasn't been released yet.
>> Again, I do not have experience with what you describe. Are there things
>> we can/should change in the pom.xml files to make things easier?
> No, this is not specific to Jena.


>>> The existing Maven plug-in for OSGi is very capable.
>> This one:
> Yes.



> Cheers,
> Dave

View raw message