lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jens Kraemer <krae...@webit.de>
Subject Re: Strategies for updating indexes.
Date Tue, 05 Apr 2005 08:29:52 GMT
Hi,
please see comments below.

On Tue, Apr 05, 2005 at 08:38:04AM +0100, Lee Turner wrote:
> Hi
> 
> I was wondering whether anyone has any experience of multithreaded
> updates to indexes.  I the web app I am working on there are additions,
> updates and deletes that need to happen to the index throughout the
> runtime of the application.  Also, the application is run in a cluster
> with each app server having its own index.  This means that periodically
> each app server is going to have to go through a re-indexing process to
> make sure that its index has all the changes from the other app servers
> in it.  This process can take a few seconds so if another update to the
> index occurs at this time it will need to be queued in some way to make
> sure it happens after the re-indexing.
> 
> I was just wondering if anyone had any pointers for doing this kind of
> thing.  Any help would be gratefully appreciated.

I usually have a service class wrapping all access to the lucene index,
which has a queue where my Servlets or Actions put the documents to be
updated or added in.  There is a single instance of this class for the
whole web app, and a thread regularly waking up and processing the
elements of the queue.

Note the queue has to be threadsafe or has to be synchronized
externally. 

Since there is only one instance of this service class, it is the only
one who will ever write to the index (if the same index is not used by
other applications).

During re-indexing the thread regularly processing the queue will
be paused. After re-indexing it is started again, processing all pending 
changes from the queue. The re-indexing itself takes place in another
thread, which is started by quartz in my case.


hope this helps you somehow,

Jens


-- 
webit! Gesellschaft für neue Medien mbH          www.webit.de
Dipl.-Wirtschaftsingenieur Jens Krämer       kraemer@webit.de
Schnorrstraße 76                      Telefon +49 351 46766 0
D-01069 Dresden                      Telefax +49 351 46766 66

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message