lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Patrick Kimber" <>
Subject Re: Lucene 2.2, NFS, Lock obtain timed out
Date Tue, 03 Jul 2007 10:17:38 GMT

I have added more logging to my test application.  I have two servers
writing to a shared Lucene index on an NFS partition...

Here is the logging from one server...

[10:49:18] [DEBUG] LuceneIndexAccessor closing cached writer
[10:49:18] [DEBUG] ExpirationTimeDeletionPolicy onCommit() delete [segments_n]

and the other server (at the same time):

[10:49:18] [DEBUG] LuceneIndexAccessor opening new writer and caching it
[10:49:18] [DEBUG] IndexAccessProvider getWriter()
[10:49:18] [ERROR] DocumentCollection update(DocumentData) I/O Error: Cannot add the
document to the index.
[/mnt/nfstest/repository/lucene/lucene-icm-test-1-0/segments_n (No
such file or directory)]

I think the exception is being thrown when the IndexWriter is created:
new IndexWriter(directory, false, analyzer, false, deletionPolicy);

I am confused... segments_n should not have been touched for 3 minutes
so why would a new IndexWriter want to read it?

Here is the whole of the stack trace: I/O Error: Cannot add the
document to the index.
[/mnt/nfstest/repository/lucene/lucene-icm-test-1-0/segments_n (No
such file or directory)]
	at lucene.icm.test.Write.add(
	at lucene.icm.test.Write.main(
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(
	at java.lang.reflect.Method.invoke(
	at org.codehaus.mojo.exec.ExecJavaMojo$
Caused by:
/mnt/nfstest/repository/lucene/lucene-icm-test-1-0/segments_n (No such
file or directory)
	at Method)
	at org.apache.lucene.index.IndexFileDeleter.<init>(
	at org.apache.lucene.index.IndexWriter.init(
	at org.apache.lucene.index.IndexWriter.<init>(
	at com.subshell.lucene.indexaccess.impl.IndexAccessProvider.getWriter(
	at com.subshell.lucene.indexaccess.impl.LuceneIndexAccessor.getWriter(
	... 13 more

Thank you very much for your previous comments and emails.

Any help solving this issue would be appreciated.


On 30/06/07, Michael McCandless <> wrote:
> Patrick Kimber wrote:
> > I have been checking the application log.  Just before the time when
> > the lock file errors occur I found this log entry:
> > [11:28:59] [ERROR] IndexAccessProvider
> >
> > /mnt/nfstest/repository/lucene/lucene-icm-test-1-0/segments_h75 (No
> > such file or directory)
> >     at Method)
> I think this exception is the root cause.  On hitting this IOException
> in reader.close(), that means this reader has not released its write
> lock.  Is it possible to see the full stack trace?
> Having the wrong deletion policy or even a buggy deletion policy (if
> indeed file.lastModified() varies by too much across machines) can't
> cause this (I think).  At worse, the wrong deletion policy should
> cause other already-open readers to hit "Stale NFS handle"
> IOExceptions during searching.  So, you should use your
> ExpirationTimeDeletionPolicy when opening your readers if they will be
> doing deletes, but I don't think it explains this root-cause exception
> during close().
> It's a rather spooky exception ... in close(), the reader initializes
> an IndexFileDeleter which lists the directory and opens any segments_N
> files that it finds.
> Do you have a writer on one machine closing, and then very soon
> thereafter this reader on a different machine does deletes and tries
> to close?
> My best guess is the exception is happening inside that initialization
> because the directory listing said that "segments_XXX" exists but then
> when it tries to open that file, it does not in fact exist.  Since NFS
> client-side caching (especially directory listing cache) is not
> generally guaranteed to be "correct", it could explain this.  But let's
> see the full stack trace to make sure this is it...
> Mike
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message