lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hes Siemelink" <hesn...@gmail.com>
Subject Re: Sudden FileNotFoundException
Date Wed, 04 Oct 2006 11:18:49 GMT
Hi Mike, thank you for your detailed reply. I put my answers inline.


On 10/4/06, Michael McCandless <lucene@mikemccandless.com> wrote:
>
> Hes Siemelink wrote:
> > It happens from time to time... but I don't know how to reproduce it.
> >
> > Rebuilding this particular index unfortunately takes about 10 hrs, so
> it's
> > not feasable to delete the index and rebuild it when this happens... our
> > users would be missing a lot of search results then!
> >
> > There are a couple of workaround strategies I could follow (using
> separate
> > indexes for updating and querying for example), but I would like to know
> > what's causing this so I can make a fix with less impact on my code.
>
> This exception typically means that the segments file is referring to
> a segment that doesn't exist in the filesytem (eg was deleted or lost
> or something).


ok, that is what I thought...
Now the question is how this file gets deleted. Should I look into Lucene
(maybe there something goes wrong while segmenting?) or into the environment
(maybe the temp directory that contains the locks is wiped, making a
critical lock disappear?)

The most common cause of this (I think) is locking not working
> properly.  Ie, a reader is opening segments at the same time that a
> writer is writing segments (and deleting old ones).  Lucene's commit
> lock exists to prevent this from happening.


Yes, updates and read occur concurrently (there is one update threads and
several search threads)

Is your application all running under one JVM (sounds like yes)?  Ie,
> in one JVM you have one thread doing updating and then multiple
> threads searching.


Yes, exactly.

What kind of filesystem is your index stored on (eg, local or NFS)?
> And how about the lock directory (normally defaults to the JVMs
> java.io.tmpdir property)?


local file system, NOT NFS.
The lock directory is default. I believe Tomcat makes it point to its own
temp directory. I'll test the possibility of putting the locks somewhere
else.

Are you sure there was no prior "root cause" exception, eg on the
> updating thread?


I could not find one...
I close the indexwriter in a finally statement in the update thread (after
each update), with a call to IndexWriter.close().

When this exception happens, does it eventually go away on its own or
> is the index permanantly corrupt (and you have to build another one)?


The index is permanently corrupt. Since the application is hosted by an
external party, I have not had the chance to inspect the files before
overwriting them.

Thanks for your help!

    Hes.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message