incubator-jena-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paolo Castagna <>
Subject Re: Fuseki build broken.
Date Tue, 10 May 2011 15:15:06 GMT
Andy Seaborne wrote:
> On 10/05/11 12:07, Paolo Castagna wrote:
>> Hi Andy,
>> I reverted the changes. More details below.
>> Andy Seaborne wrote:
>>> On 10/05/11 11:15, Paolo Castagna wrote:
>>>> Hi Andy
>>>> Andy Seaborne wrote:
>>>>> Fuseki does not currently compile and some other things have changed
>>>>> without warning in such a way a build would affect a user:
>>>>> I would prefer that systems are always in a state whereby they can be
>>>>> built and released unless they is good reason and notification.
>>>> Yes, I agree.
>>>>> 1/ Depends on a non-existent TDB 0.8.11-SNAPSHOT
>>>> Sorry, I didn't spot this immediately (since I had TDB 0.8.11-SNAPSHOT
>>>> in my
>>>> local Maven repository).
>>>> Will publishing a TDB 0.8.11-SNAPSHOT fix the problem?
>>> But there are a mix of outstanding patches for ARQ and TDB. What state
>>> is Fuseki in?
>> If Fuseki depends on a SNAPSHOT (either ARQ or TDB) is in the same state
> Which is?  They have some sort of dependency on LARQ but I don't know 
> what it is.

SNAPSHOT == in development.

ARQ (includes the old/legacy LARQ) but it does not depend on LARQ.
LARQ depends on ARQ, instead. This is to avoid cyclic dependencies.
We use reflection to wire the new LARQ into ARQ (if present in the

TDB does not depend on LARQ.

If someone wants to use the new LARQ (the one distributed as separate
module) they need to have their project depending on ARQ and on LARQ
(and, at the moment, they will need to exclude dependency on Lucene
v2.3.1 from ARQ).

>> To release it, it must not depend on SNAPSHOTs so, either a dependency
>> on a SNAPSHOT is decreased to the latest stable release or a release of
>> the dependent module is necessary first.
>> When is the next Fuseki release planned?
> Bug fix releases are not "planned".  It is, to me, rather important the 
> system is buildable and releasable at all times possible.

Yes. I agree.

> People do use development builds out of SVN.  We even ask people to test 
> thing by doing that.


>>>> However, at release time Fuseki will need to depend only on non
>>>> SNAPSHOT artifacts though.
>>>>> 2/ Introduces a dependency on talis-oss-snapshots Maven repository.
>>>> I'll remove this one.
>>>> Is it possible to have a SNAPSHOT for LARQ on the Jena's repo-dev?
>>> Yes - I have to do it because the file permissions are set up right
>>> and I've never got round to fixing it.
>> It would be useful to have a LARQ SNAPSHOT on jena-dev, this way we can
>> try it or depend on it (from SNAPSHOT artifacts) without introducing
>> dependencies on other repositories.
>> I still need to check if it is possible (and how) for incubating projects
>> to use:
>> All committers should be able to publish SNAPSHOTs or a CI systems
>> should/could do this for us (every commit or daily).
>>> But this should not matter
>> It removes the dependency on talis-oss-snapshots Maven repository.
>>>>> 3/ Updating the dependencies does not get a working development 
>>>>> system.
>>>> Could you expand on this?
>>> I updated Fuseki - it does not compile even after updating the
>>> dependencies.
>> What was your problem?
>> Was it from Maven or from Eclipse?
>> It was compiling for me, both from the command line and Eclipse.
>> I was able to package a new Fuseki artifacts, run it and test LARQ was
>> functioning correctly. Indeed, this is what I did before committing
>> any change.
>> But, as I said, I had TDB 0.8.11-SNAPSHOT (and an updated
>> ARQ-2.8.9-SNAPSHOT)
>> in my local Maven repo and therefore I did not see the problem.
> The artifact in the Talis repo is not called 0.8.11-SNAPSHOT.

We use the talis-snapshots repo to publish SNAPSHOTs with uncommitted
patches we want to test and allow/invite other people to test.

The artifacts there without a JIRA issue number and without SNAPSHOT
are not a good idea (and indeed a mistake, fixed now).

>>>>> It also uses LARQ and seems to force Lucene 3 - isn't that a one-way
>>>>> migration isn't it?
>>>> LARQ in ARQ depends on Lucene v2.3.1
>>>> LARQ as separate module depends on Lucene v3.1.0.
>>> Yes - but what is the migration plan for LARQ?
>>> Is the next release of ARQ going to have just separate LARQ or a
>>> mixture of internal and external modules? (seems kind of complicated).
>>> When does this happen?
>> To minimize problems and changes, I propose to release ARQ as it is
>> with the old legacy LARQ depending on Lucene 2.3.1 included.
>> However, I propose to commit the changes discussed in JENA-63 so that
>> it is possible and easy to enable and use LARQ (old or new from Fuseki
>> or other systems using TDB).
>> We could add the new LARQ (as separate module) to Fuseki.
>> In order to do this, Fuseki needs to depend on LARQ as well as on a
>> version of ARQ which includes the necessary changes to the
>> DatasetAssembler.
>> This will make easy for users to enable and use LARQ with Fuseki.
>> They'll just need to add a line to their assembler config file.
>> Does this seem reasonable to you?
>> Having LARQ is Fuseki is really valuable IMHO.
>> Using the new LARQ keeps Lucene indexes up-to-date with TDB and it avoids
>> duplicates. This, IMHO, is another really valuable thing for LARQ users.
> Not disagreeing but there seems to be no migration plan or documentation.

If we leave the old/legacy LARQ in ARQ as it is, no migration plan is necessary.
People can decide which one to use.

Documentation: yes, necessary but there are no actual changes from an API or
use point of view. For developers, I'll document how to add a dependency on
LARQ artifacts if they want to use the new/separate module.

> Why don't we just switch to LARQ now?

I'd prefer to add LARQ to Fuseki first, so that we can make it easier for
people to try it out, they can report us feedback and if it's ok we can
then remove the old LARQ from ARQ.

Leaving LARQ in ARQ minimize risk and changes and it offers a fall back
to people.

> If it's not automatic (Lucene3 can read Lucen2 indexes?), then we need 
> to document how the user can do an upgrade.

The new LARQ adds a new field to the Lucene index (i.e. "hash") to support
unindex/deletes, see [1], therefore a reindex (using the usual larqbuild is


>     Andy
>> Paolo
>>>> Some methods have been deprecated and then removed from Lucene v3.x,
>>>> therefore it is not possible to have a system using Lucene v3.x which
>>>> allows for a drop-in replacement if someone needs to an older version
>>>> of Lucene.
>>>> I can revert the changes, but having LARQ available in Fuseki is really
>>>> valuable for me (and I suspect many other Fuseki users).
>>> See above on migration plan. Basically, what is going on? Suppose a
>>> serious bug comes up in ARQ? TDB? Fuseki? can it be fixed and released
>>> now?
>> If a serious bug comes up in ARQ, it can be fixed a a new bug fix release
>> can be published for ARQ. ARQ currently compiles and all the tests pass.
>> If a serious bug comes up in TDB, same as above. Fix => bug fix release.
>> TDB currently compiles and all tests pass.
>> For Fuseki, since it depends on ARQ and TDB, there are two different
>> situations:
>> 1. Fuseki depends on ARQ or TDB SNAPSHOTs
>> 2. Fuseki depends on ARQ or TDB latest releases
>> In case of 1. if Fuseki needs the changes in ARQ or TDB SNAPSHOTs, a
>> release
>> of Fuseki must be preceded by a release of ARQ or TDB.
>>>> Paolo
>>> Andy

View raw message