lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bugzi...@apache.org
Subject DO NOT REPLY [Bug 32171] - Lock obtain time out errors when opening readers and writers
Date Tue, 30 Nov 2004 01:19:41 GMT
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG·
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
<http://issues.apache.org/bugzilla/show_bug.cgi?id=32171>.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND·
INSERTED IN THE BUG DATABASE.

http://issues.apache.org/bugzilla/show_bug.cgi?id=32171





------- Additional Comments From hossman_apachebugz@fucit.org  2004-11-30 02:19 -------
For what it's worth, I can definitely reproduce this on my debian linux box...

Linux asimov 2.6.6.hoss1 #1 SMP Tue Jul 6 16:31:01 PDT 2004 i686 GNU/Linux

...it allways crashes within 30 seconds.

The System.out info seems to indicate that first the IndexReader thread is doing
a few thousand iterations, then the IndexWriter thread is doing a few thousand
iterations, then the IndexReader thread jugs along for many thousands of
iterations without any interuptions, untill the IndexWriter thread generates the
lock exception...

java.io.IOException: Lock obtain timed out:
Lock@/tmp/lucene-b5cc3e5c3fb36ba171e16cf179c80def-commit.lock
        at org.apache.lucene.store.Lock.obtain(Lock.java:58)
        at org.apache.lucene.store.Lock$With.run(Lock.java:108)
        at org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:501)
        at
org.apache.lucene.index.IndexWriter.flushRamSegments(IndexWriter.java:440)
        at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:242)
        at TestLuceneLocks$2.run(TestLuceneLocks.java:63)
        at java.lang.Thread.run(Thread.java:534)
     [java] Total time: 17secs


in a dozen tests, the stack trace was always identicle except for the lock filename.

(NOTE: If you modify the test case to use a RAMDirectory, it seems to run
indefinitely, which would suggest to me that the issue definitely likes in the
difference between the Lock instances returned by Directory.makeLock)

I'm not really a concurrency expect, but I'm guessing that the problem has to do
with the way Lucene does locking.  It loks like whenever something wants an
exclusive lock, Lock.obtaion(long) will sleep in 1 second intervals waking up
each escond to see if "now" it can obtain() the lock.  Which means that if some
other client (in this case the IndexReader thread) is constantly accessing the
index, it's very easy for a situation to arise in which the thread scheduler is
constantly letting the IndexWritter thread (which is waiting on the lock) "wake
up" while the reader is still blocking it.

I seem to recall that the idiom of "while (!ready) { sleep() }" is considered
not the best aproach, and that using "wait()" instead of sleep is considered
better ... but i don't remember exactly why, (or even if i'm right about which
one is better).   So i think that it might be possible to change lucene so that
it doesn't exhibit this problem -- but i'm not certain whatthat change might be.

In general, it seems like you might be able to work around this problem, by
making your "reader" thread be a little more considerate, and making your writer
thread retry a couple of times if it gets a lock exception.




-- 
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org


Mime
View raw message