lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hes Siemelink" <hesn...@gmail.com>
Subject Don't use the same index for updating and searching
Date Thu, 19 Oct 2006 14:37:47 GMT
Hi, I posted a while ago about sudden FileNotFoundExceptions. In a nutshell:
my Lucene index went corrupt after a couple of days under heavy load on a
Linux server with missing segment files.

The problem kept occuring, but I haven't found the cause. I couldn't
reproduce it with a simulated load on my development machine or the test
server.

Here's a sketch of the setup: there were 14 indexes that received
simultaneous updates and search queries. Each index had own thread for
updates (so the index wasn;t updated concurrently), but allowed for reading
from other threads. Updating consisted of deleting on index intry and adding
a new one. On average I would say there was one update request and one
search request per second.

So, this turned out not to be stable! After a while (a day or so, or two),
any index would get corrupted, because a segment would disappear.

I circumvented the problem using seperate indexes for searching and
updating. The search index is copied every ten minutes to accomodate new
search requests. The problem seems to have gone away.

So here's my tip: if you have continuous updates and a lot of search
requests, use different indexes for searching and for updating!

Cheers,

   Hes.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message