lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From robert engels <reng...@ix.netcom.com>
Subject Re: 15 minute hang in IndexInput.clone() involving finalizers
Date Fri, 15 Dec 2006 16:58:42 GMT
If you could post a complete thread dump that would be better.

I am "thinking" that it is not really a bug, but that the finalizer  
or GC thread is very busy, or possibly blocked on a network resource,  
and that is preventing the addition of additional finalizers.

(sorry for the double post but it belongs here).

On Dec 15, 2006, at 12:28 AM, Chuck Williams wrote:

> Hi All,
>
> I've had a bizarre anomaly arise in an application and am wondering if
> anybody has ever seen anything like this.  Certain queries, in not  
> easy
> to reproduce cases, take 15-20 minutes to execute rather than a few
> seconds.  The same query is fast some times and anomalously slow
> others.  This is on a 1,000,000 document collection, but the problem
> seems independent of that.
>
> I took a bunch of thread dumps during the anomaly period.  There are 4
> threads executing the same query at the same time, and all 4 appear to
> spend almost the entire time trying to register finalizers as part of
> cloning an IndexInput within an application call to create a
> TopFieldDocCollector into which the results will be collected.  The
> actual search has not been launched yet, and will be reasonably quick
> when it is.  All 4 threads show this unchanging stack trace during the
> 15-20 minutes:
>
>>> == Thread Connection thread group.HttpConnection-26493-11 ===>
>>> java.lang.ref.Finalizer.add(Unknown Source)
>>>         java.lang.ref.Finalizer.<init>(Unknown Source)
>>>         java.lang.ref.Finalizer.register(Unknown Source)
>>>         java.lang.Object.clone(Native Method)
>>>         org.apache.lucene.store.IndexInput.clone(IndexInput.java: 
>>> 175)
>>>
>>> org.apache.lucene.store.BufferedIndexInput.clone 
>>> (BufferedIndexInput.java:128)
>>>         org.apache.lucene.store.FSIndexInput.clone 
>>> (FSDirectory.java:562)
>>>
>>> org.apache.lucene.index.SegmentTermDocs.<init> 
>>> (SegmentTermDocs.java:45)
>>>
>>> org.apache.lucene.index.SegmentReader.termDocs(SegmentReader.java: 
>>> 333)
>>>
>>> org.apache.lucene.index.MultiTermDocs.termDocs(MultiReader.java:416)
>>>
>>> org.apache.lucene.index.MultiTermDocs.termDocs(MultiReader.java:409)
>>>         org.apache.lucene.index.MultiTermDocs.next 
>>> (MultiReader.java:361)
>>>
>>> org.apache.lucene.index.ParallelReader$ParallelTermDocs.next 
>>> (ParallelReader.java:353)
>>>
>>> org.apache.lucene.search.FieldCacheImpl$3.createValue 
>>> (FieldCacheImpl.java:173)
>>>
>>> org.apache.lucene.search.FieldCacheImpl$Cache.get 
>>> (FieldCacheImpl.java:72)
>>>
>>> org.apache.lucene.search.FieldCacheImpl.getInts 
>>> (FieldCacheImpl.java:154)
>>>
>>> org.apache.lucene.search.FieldCacheImpl.getInts 
>>> (FieldCacheImpl.java:148)
>>>
>>> org.apache.lucene.search.FieldSortedHitQueue.comparatorInt 
>>> (FieldSortedHitQueue.java:204)
>>>
>>> org.apache.lucene.search.FieldSortedHitQueue$1.createValue 
>>> (FieldSortedHitQueue.java:175)
>>>
>>> org.apache.lucene.search.FieldCacheImpl$Cache.get 
>>> (FieldCacheImpl.java:72)
>>>
>>> org.apache.lucene.search.FieldSortedHitQueue.getCachedComparator 
>>> (FieldSortedHitQueue.java:155)
>>>
>>> org.apache.lucene.search.FieldSortedHitQueue.<init> 
>>> (FieldSortedHitQueue.java:56)
>>>
>>> org.apache.lucene.search.TopFieldDocCollector.<init> 
>>> (TopFieldDocCollector.java:41)
> ... application stack
>
> Another factor appears to be that this anomaly usually (maybe always)
> happens just after a series of deleteAdd updates, i.e. just after a
> series of deleting with the IndexReader, closing it to add a modified
> version of that document with the IndexWriter, and then reopening the
> IndexReader.  A query just after reopening the IndexReader is most
> likely to trigger this issue.
>
> I have not seen this problem on any other collecitons with the same
> application, and so it may be specific to this collection or to its  
> size.
>
> Any thoughts or ideas would be appreciated.
>
> Thanks,
>
> Chuck
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message