lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chengdu Huang <chengdu.hu...@patterninsight.com>
Subject Re: deadlock in indexing
Date Tue, 28 Jul 2009 21:10:51 GMT
Hi Simon,

By "IndexWriter is threadsafe" do you mean that I can have to two
threads, one calls IndexWriter.addDocument(), the other calls
IndexWriter.deleteDocuments() & IndexWriter.optimize(), without any
synchronization?

For the deadlock, I also think it has something to do with merging.
Below are stacktraces of some of the relevant threads:

"Lucene Merge Thread #2" daemon prio=10 tid=0x086be000 nid=0x4a9
waiting for monitor entry [0x52dad000..0x52dadf20]
   java.lang.Thread.State: BLOCKED (on object monitor)
        at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:3964)
        - waiting to lock <0x58a745d0> (a org.apache.lucene.index.IndexWriter)
        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3872)
        at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205)
        at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260)

"Lucene Merge Thread #1" daemon prio=10 tid=0x517c3400 nid=0x4a7
waiting for monitor entry [0x52dfe000..0x52dfeea0]
   java.lang.Thread.State: BLOCKED (on object monitor)
        at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:3964)
        - waiting to lock <0x58a745d0> (a org.apache.lucene.index.IndexWriter)
        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3872)
        at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205)
        at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260)

"Lucene Merge Thread #0" daemon prio=10 tid=0x08714c00 nid=0x4a1
waiting for monitor entry [0x52ff7000..0x52ff7e20]
   java.lang.Thread.State: BLOCKED (on object monitor)
        at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:3964)
        - waiting to lock <0x58a745d0> (a org.apache.lucene.index.IndexWriter)
        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3872)
        at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205)
        at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260)

"Add Document Thread" prio=10 tid=0x084ef000 nid=0x4985 in
Object.wait() [0x5315c000..0x5315d020]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x58af4548> (a
org.apache.lucene.index.ConcurrentMergeScheduler)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.lucene.index.ConcurrentMergeScheduler.merge(ConcurrentMergeScheduler.java:182)
        - locked <0x58af4548> (a
org.apache.lucene.index.ConcurrentMergeScheduler)
        at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2402)
        at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2397)
        at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2393)
        at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3443)
        at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1922)
        at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1880)
        at com.patterninsight.indexing.IndexTask.indexDocuments(IndexTask.java:143)
        - locked <0x58b422c0> (a org.apache.lucene.index.IndexWriter)

It seems that the last thread has locked the ConcurrentMergeScheduler,
but then wait on it too.  Is that the problem?

Chengdu

On Tue, Jul 28, 2009 at 12:28 AM, Simon
Willnauer<simon.willnauer@googlemail.com> wrote:
> I can not help you to figure out your exact problem but you can use an
> the same indexwriter instance without synchronization. IndexWriter is
> threadsafe so you synchronized block seems obsolet.
> I could imagine that there is a backgroud merge going on while you try
> to access the critical section ( you synchronized block) which could
> block you code for a while until the merge has finished. Can you
> figure out if a merger-thread is running? The threads name should be
> set to something like "Lucene Merge Thread #n"
>
> simon
>
> On Tue, Jul 28, 2009 at 6:27 AM, Chengdu
> Huang<chengdu.huang@patterninsight.com> wrote:
>> Hi,
>>
>> I have an application in which documents are added upon receiving a
>> user request and a background thread is needed to remove old
>> documents.  I have an IndexWriter opened on a Directory that adds
>> documents and commits but never closes.  The background thread that
>> removes documents uses the same instance of IndexWriter.  So the code
>> looks like
>>
>> // Thread to add document:
>> synchronized(writer) {
>>  try {
>>    Document doc = new Document();
>>    doc.add();
>>    ...
>>    writer.commit();
>>  } catch (Exception e) {
>>    writer.rollback();
>>  }
>> }
>>
>> Now looks like I run into some kind of deadlock here even *WITHOUT*
>> the background thread of removing documents.  The symptom is that the
>> whole java process is on sleeping state and jstack shows that the
>> thread to add document is blocked on waiting an object.  Unfortunately
>> I'm unable to reproduce this in unittests.
>>
>> My guess is that the outer synchronized(writer) {} block is causing
>> the problem, but can't figure out why.  Any idea?
>>
>> Chengdu
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message