lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher Tignor <ctig...@thinkmap.com>
Subject Re: IndexWriter optimize() deadlock
Date Fri, 16 Oct 2009 19:17:58 GMT
It doesn't look like my Future.cancel(true) is actually interrupting the
thread.  It only does so "if necessary" and in this case seems to be letting
the Thread finish gracefully without need for interruption.

The stack trace leading up to the hanging IndexWriter.optimize() method is
below, though not terribly useful I imagine.  Here only 5 documents are
trying to be added to an index taking up only 39.2 MB on disk.  Again this
deadlocking only happens after I use the Future for this task to cancel the
task...

Thread [pool-3-thread-5] (Suspended)
    IndexWriter.doWait() line: 4494
    IndexWriter.optimize(int, boolean) line: 2283
    IndexWriter.optimize(boolean) line: 2218
    IndexWriter.optimize() line: 2198
    LuceneResultsPersister.commit() line: 97
    PersistenceJobQueue.persistAndCommitBatch(PersistenceJobConsumer) line:
105
    PersistenceJobConsumer.consume() line: 46
    PersistenceJobConsumer.run() line: 67
    Executors$RunnableAdapter<T>.call() line: 441
    FutureTask$Sync.innerRun() line: 303
    FutureTask<V>.run() line: 138
    ThreadPoolExecutor$Worker.runTask(Runnable) line: 886
    ThreadPoolExecutor$Worker.run() line: 908
    Thread.run() line: 619

thanks,

C>T>

On Fri, Oct 16, 2009 at 1:58 PM, Michael McCandless <
lucene@mikemccandless.com> wrote:

> My guess is it's the invocation of Thread.interrupt (which
> Future.cancel(true) calls if the task is running) that lead to the
> deadlock.
>
> Is it possible to get the stack trace of the thrown exception when the
> thread was interrupted?  Maybe indeed something in IW isn't cleaning
> up its state on being interrupted.
>
> Mike
>
> On Fri, Oct 16, 2009 at 1:43 PM, Christopher Tignor
> <ctignor@thinkmap.com> wrote:
> > thanks for getting back.
> >
> > I do not lock on the IndexWriter object itself but all methods in my
> > consumer class that use IndexWriter are synchronized (locking my
> singleton
> > consumer object itself).
> > The thread is waiting at IndexWriter.doWait().  What might cuase this?
> >
> > thanks -
> >
> > C>T>
> >
> > On Fri, Oct 16, 2009 at 12:58 PM, Uwe Schindler <uwe@thetaphi.de> wrote:
> >
> >> Do you use the IndexWriter as mutex in a synchronized() block? This is
> not
> >> supported and may hang. Never lock on IndexWriter instances. IndexWriter
> >> itself is thread safe.
> >>
> >> -----
> >> Uwe Schindler
> >> H.-H.-Meier-Allee 63, D-28213 Bremen
> >> http://www.thetaphi.de
> >> eMail: uwe@thetaphi.de
> >>
> >> > -----Original Message-----
> >> > From: Christopher Tignor [mailto:ctignor@thinkmap.com]
> >> > Sent: Friday, October 16, 2009 6:50 PM
> >> > To: java-user
> >> > Subject: IndexWriter optimize() deadlock
> >> >
> >> > Hello,
> >> >
> >> > I am trying to track down the cause of my code hanging on calling
> >> > IndexWriter.optimize() at its doWait() method.
> >> > It appears, thus that it is watiing on other merges to happen which is
> a
> >> > bit
> >> > confusing to me:
> >> >
> >> > My application is a simple producer consumer model where documents are
> >> > added
> >> > to a queue by producers and then one consumer with one indexwriter
> (the
> >> > only
> >> > in the application) periodically calls addDocument() on a batch of
> these
> >> > jobs and then calls optimize(), commit(). and then close().  There is
> >> only
> >> > one thread running the consumer so I am confused as to how the
> >> indexwriter
> >> > might be deadlocking itself.  Indeed this is the only thread active
> when
> >> > the
> >> > deadlock occurs so it seems to be a problem of reentry.
> >> >
> >> > Importantly, the deadlocking occurs only when the thread is trying to
> >> > shutdown - that is the Thread running this lucene consumer has a
> Future
> >> > that
> >> > has had its cancel(true) interrupting method called.  Is it possible
> that
> >> > an
> >> > internal Lucene lock is obtained during addDocument() and on
> interruption
> >> > is
> >> > never released so the subsequent optimize() call hangs?  This doesn't
> >> > appear
> >> > to be happening...
> >> >
> >> > Any help appreciated.
> >> >
> >> > thanks,
> >> >
> >> > C>T>
> >> >
> >> > what might I be missing here?
> >> >
> >> > --
> >> > TH!NKMAP
> >> >
> >> > Christopher Tignor | Senior Software Architect
> >> > 155 Spring Street NY, NY 10012
> >> > p.212-285-8600 x385 f.212-285-8999
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >>
> >
> >
> > --
> > TH!NKMAP
> >
> > Christopher Tignor | Senior Software Architect
> > 155 Spring Street NY, NY 10012
> > p.212-285-8600 x385 f.212-285-8999
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>


-- 
TH!NKMAP

Christopher Tignor | Senior Software Architect
155 Spring Street NY, NY 10012
p.212-285-8600 x385 f.212-285-8999

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message