lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Wang" <>
Subject realtime indexing
Date Fri, 16 Nov 2007 06:44:22 GMT

    It was interesting hearing about the need for real time indexing
at the BirdsOfAFeather round table. We also needed to solve this
problem. We took this approach:

A large disk index that indexes in batch, e.g. sleeps for some time
queue up requests, wakes up and the index.
While large disk index is sleeping, same requests are also added to a
ram index, and when disk indexer is working, requests received is
added to another ram index.

When new disk index is published, the first ram index points to the
secondary ram index, and the secondary ram index is flushed.

we keep 1 index reader open for the disk index, and create new
indexReaders for the ram indexes per request (it seems to be ok
because the ram indexes are small)

We use MultiSearcher across these readers.

duplicates are also handled with our scheme.

I am curious to see if anyone else is trying this. It would be
interesting to hear comments from the experts.



To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message