archiva-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martin <marti...@apache.org>
Subject Re: maven-indexer / Lucene
Date Sat, 24 Jun 2017 20:36:15 GMT
I think we merge the branch to the master, when it's decided what way to go.

Do you refer to the intermittent failures of store-jcr module with your answer 
or do you refer to other issues?
I had the intermittent failures too and thought about some race condition, 
because JCR Oak is very asynchronous by design.

Greetings

Martin

Am Samstag, 24. Juni 2017, 01:50:40 CEST schrieb Olivier Lamy:
> well the issue is non compatible version of Lucene for Maven Indexer and
> Oak (well I can try push a patch to Oak for upgrading...)
> 
> On 24 June 2017 at 08:41, Olivier Lamy <olamy@apache.org> wrote:
> > Hi
> > Maven Indexer 6.0-SNAPSHOT doesn't need anymore plexus bridge.
> > I'm working on it in the branch ( feature/jcr_oak )
> > Not sure why but I have intermittent failure with store-jcr module.
> > I definitely agree on the upgrade.
> > Well we can simply detect it's not oak compatible and schedule a full
> > reindex (maybe with a message in logs and ui?)
> > But we need to be sure we can still read central index and not sure about
> > possible lucene conflict with oak and maven indexer.
> > We can work on this branch? (I created a Jenkins job for it
> > https://builds.apache.org/view/A-D/view/Archiva/job/
> > archiva-jcr-oak-branch/)
> > If you prefer master I would say no worries neither.
> > Something else to look at is upgrading maven-core etc...
> > Anyway
> > Cheers
> > Olivier
> > 
> > On 22 June 2017 at 19:16, Martin <martin_s@apache.org> wrote:
> >> Hi,
> >> 
> >> upgrading the maven indexer leads to some major changes.
> >> Lucene is used by maven-indexer and also by jackrabbit. Jackrabbit sticks
> >> to
> >> the old 3.x version and, as I see it, they will not move to a newer
> >> version.
> >> There is Jackrabbit Oak as alternative.
> >> I tried a proof of concept and could replace the jackrabbit
> >> implementation of
> >> metadata-store-jcr with a oak implementation. At least I got the unit
> >> tests of
> >> this module all to pass.
> >> But switching to Oak has some drawbacks:
> >> - The repository format changed and we must provide a way to migrate
> >> (either
> >> migrate the existing repository or create a new one by reindexing)
> >> - The lucene version used is newer but does not match to the version from
> >> the
> >> maven-indexer dependencies. There may come up some incompatibilities that
> >> are
> >> not solvable without using a modified version of one of the both. Or
> >> there may
> >> be the possibility to switch to solr (as separate component) and get rid
> >> of
> >> the lucene dependencies for jcr inside the archiva project.
> >> 
> >> Switching to maven-indexer 6.0-SNAPSHOT means some changes too:
> >> - The Plexus-Sisu-Bridge does not work as before.
> >> - We must migrate from the NexusIndexer to the indexer API.
> >> 
> >> So switching to the new indexer and oak means more work as expected and
> >> some
> >> risks regarding new incompatibility problems. And I think this cannot be
> >> done
> >> without broken master builds for some time period.
> >> 
> >> So, what should we do? I think maven indexer is one of the core
> >> components of
> >> archiva, and we should utilize the 3.x-version to  migrate to the new
> >> indexer
> >> version, even if this means switching to jcr oak. Otherwise it would mean
> >> to
> >> stick to the old version for the next years.
> >> @Olivier, regarding the maven-indexer / sisu-Bridge API changes, I hope
> >> you
> >> can provide  useful help.
> >> 
> >> I committed the PoC to the branch feature/jcr_oak. There are some modules
> >> where the tests do not pass (mainly because of the indexer API changes).
> >> 
> >> Any comments?
> >> 
> >> Cheers
> >> 
> >> Martin
> >> 
> >> Am Dienstag, 13. Juni 2017, 09:07:35 CEST schrieb Olivier Lamy:
> >> > forget it but we need to ensure we can read maven index files....
> >> > 
> >> > On 13 June 2017 at 17:06, Olivier Lamy <olamy@apache.org> wrote:
> >> > > Hi,
> >> > > Remember jackrabbit depends on Lucene as well so upgrading Lucene
can
> >> 
> >> be a
> >> 
> >> > > problem here.
> >> > > Regarding maven-indexer yes we can depend on a snapshot until the
> >> 
> >> release.
> >> 
> >> > > I can release it ;-)
> >> > > 
> >> > > On 13 June 2017 at 06:06, Martin <martin_s@apache.org> wrote:
> >> > >> Hi,
> >> > >> 
> >> > >> the lucene version depends on the maven indexer. But I'm not sure
> >> 
> >> about
> >> 
> >> > >> the
> >> > >> current state of maven-indexer. The version has not changed since
> >> 
> >> some
> >> 
> >> > >> 2013.
> >> > >> 
> >> > >> There are commits on the master branch since then, and the lucene
> >> 
> >> version
> >> 
> >> > >> has
> >> > >> been changed too, but no releases were tagged.
> >> > >> Does it make sense to switch to the maven-indexer 6.0-SNAPSHOT?
> >> > >> 
> >> > >> As I know there are new compact index formats with new lucene
> >> 
> >> versions
> >> 
> >> > >> but I'm
> >> > >> not sure if this is relevant for the maven indexes.
> >> > >> 
> >> > >> Cheers
> >> > >> 
> >> > >> Martin
> >> > > 
> >> > > --
> >> > > Olivier Lamy
> >> > > http://twitter.com/olamy | http://linkedin.com/in/olamy
> > 
> > --
> > Olivier Lamy
> > http://twitter.com/olamy | http://linkedin.com/in/olamy



Mime
View raw message