lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jens Kraemer <krae...@webit.de>
Subject Re: Strategies for updating indexes.
Date Wed, 06 Apr 2005 12:06:40 GMT
On Tue, Apr 05, 2005 at 08:16:35AM -0700, Otis Gospodnetic wrote:
> If you take this approach, keep in mind that you will also need to
> handle regular application shutdowns, and also try to catch some
> crashes/errors, in order to flush your in-memory queue of items
> scheduled for indexing, and write them to disk.

right, when complete index rebuilds are expensive and therefore seldom
or never take place this is essential.

> Feel free to post the code, if you want and can, so people don't have
> to reinvent this.

I implemented the solution I mentioned in a lucene indexing plugin I
wrote for XWiki (xwiki.org). It doesn't have any persistency solution
for the queue to preserve state of the queue during application
downtime, but relies on complete index rebuilds to be done after such
events.

code (GPL) and docs can be found at
http://www.jkraemer.net/maven/xwiki/plugins/lucene-plugin/

The relevant piece of code is the IndexUpdater class which holds a
XWikiDocumentQueue and implements several XWiki interfaces to receive
notifications when documents change. IndexUpdater#run is executed in
it's own thread (spawned by the LucenePlugin class), regularly checking
the queue and updating the index when necessary.

Index rebuilds are done by emptying the index and then feeding all
documents into the updater's queue. At least on unix systems this works
fine, searches during the rebuild are still done using the old IndexReader 
instance which was opened when the index still was there. After rebuild 
is finished, the reader is closed and a new one is opened, now searching 
takes place on the new index.

Jens


-- 
webit! Gesellschaft für neue Medien mbH          www.webit.de
Dipl.-Wirtschaftsingenieur Jens Krämer       kraemer@webit.de
Schnorrstraße 76                      Telefon +49 351 46766 0
D-01069 Dresden                      Telefax +49 351 46766 66

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message