lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Re: Strategies for updating indexes.
Date Tue, 05 Apr 2005 15:16:35 GMT
If you take this approach, keep in mind that you will also need to
handle regular application shutdowns, and also try to catch some
crashes/errors, in order to flush your in-memory queue of items
scheduled for indexing, and write them to disk.

Feel free to post the code, if you want and can, so people don't have
to reinvent this.

Otis


--- Jens Kraemer <kraemer@webit.de> wrote:
> Hi,
> please see comments below.
> 
> On Tue, Apr 05, 2005 at 08:38:04AM +0100, Lee Turner wrote:
> > Hi
> > 
> > I was wondering whether anyone has any experience of multithreaded
> > updates to indexes.  I the web app I am working on there are
> additions,
> > updates and deletes that need to happen to the index throughout the
> > runtime of the application.  Also, the application is run in a
> cluster
> > with each app server having its own index.  This means that
> periodically
> > each app server is going to have to go through a re-indexing
> process to
> > make sure that its index has all the changes from the other app
> servers
> > in it.  This process can take a few seconds so if another update to
> the
> > index occurs at this time it will need to be queued in some way to
> make
> > sure it happens after the re-indexing.
> > 
> > I was just wondering if anyone had any pointers for doing this kind
> of
> > thing.  Any help would be gratefully appreciated.
> 
> I usually have a service class wrapping all access to the lucene
> index,
> which has a queue where my Servlets or Actions put the documents to
> be
> updated or added in.  There is a single instance of this class for
> the
> whole web app, and a thread regularly waking up and processing the
> elements of the queue.
> 
> Note the queue has to be threadsafe or has to be synchronized
> externally. 
> 
> Since there is only one instance of this service class, it is the
> only
> one who will ever write to the index (if the same index is not used
> by
> other applications).
> 
> During re-indexing the thread regularly processing the queue will
> be paused. After re-indexing it is started again, processing all
> pending 
> changes from the queue. The re-indexing itself takes place in another
> thread, which is started by quartz in my case.
> 
> 
> hope this helps you somehow,
> 
> Jens
> 
> 
> -- 
> webit! Gesellschaft für neue Medien mbH          www.webit.de
> Dipl.-Wirtschaftsingenieur Jens Krämer       kraemer@webit.de
> Schnorrstraße 76                      Telefon +49 351 46766 0
> D-01069 Dresden                      Telefax +49 351 46766 66
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message