jackrabbit-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ard Schrijvers <a.schrijv...@onehippo.com>
Subject Re: Adding new nodes to a cluster
Date Thu, 16 Sep 2010 07:57:39 GMT
On Thu, Sep 16, 2010 at 1:11 AM, Ian Boston <ieb@tfd.co.uk> wrote:
> On 14 Sep 2010, at 21:14, Ard Schrijvers wrote:
>> Current jackrabbit lucene architecture also doesn't fit something like
>> infinispan LuceneDirectory (it needs a reopen() on every call), which
>> would be a very nice thing to be able to use. Anyway, tons of ideas I
>> have, all I need is some (much) time :-(((
> Ard,
> This might be dangerous talk,
> Since Infinispan covers both distributed caching and on top of that the management of
Lucene indexing, how hard do you think it would be to replace all of the areas the Journal
processing touches with Infinispan?

As far as it is concerned with Lucene, we wouldn't need anything like
a journal any more (I think the journal is also used for cache
eviction, that would then be the use case for it only I guess though i
am on thin ice here as it is not my expertise): Every node gets a
clustered in memory Lucene index. Currently, the jackrabbit Lucene
architecture is not compatible with this however. Also, the Lucene
indexes become way to big (at the moment).

> I have a suspicion that Jackrabbit use of the lucene index may write too often for the
Infinispan implementation to work, however thats just a suspicion, nothing more.

How often is written I don't suspect it to be an issue for infinispan:
Infinispan has a LuceneDirectory build on top of it. I think they
target environments with many more writes then Jackrabbit usage on
average does. I recently talked to an active infinispan contributor
who is also a Hibernate Search contributor, making use of infinispan
in a clustered hibernate solution: their usecase is very much alike

Anyway, for this to be possible, we would need to do quite some
reshuffling in the existing Lucene...starting with reopen() a single
index reader for every request instead of the current logic with many
index readers which are kept open forever. More about this at some
later point, I really hope I can move time into this area. Not this
year any more I think however :-(

Regards Ard

> Ian

View raw message