lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chuck Williams <ch...@manawiz.com>
Subject 15 minute hang in IndexInput.clone() involving finalizers
Date Fri, 15 Dec 2006 06:28:47 GMT
Hi All,

I've had a bizarre anomaly arise in an application and am wondering if
anybody has ever seen anything like this.  Certain queries, in not easy
to reproduce cases, take 15-20 minutes to execute rather than a few
seconds.  The same query is fast some times and anomalously slow
others.  This is on a 1,000,000 document collection, but the problem
seems independent of that.

I took a bunch of thread dumps during the anomaly period.  There are 4
threads executing the same query at the same time, and all 4 appear to
spend almost the entire time trying to register finalizers as part of
cloning an IndexInput within an application call to create a
TopFieldDocCollector into which the results will be collected.  The
actual search has not been launched yet, and will be reasonably quick
when it is.  All 4 threads show this unchanging stack trace during the
15-20 minutes:

>> == Thread Connection thread group.HttpConnection-26493-11 ===>
>> java.lang.ref.Finalizer.add(Unknown Source)
>>         java.lang.ref.Finalizer.<init>(Unknown Source)
>>         java.lang.ref.Finalizer.register(Unknown Source)
>>         java.lang.Object.clone(Native Method)
>>         org.apache.lucene.store.IndexInput.clone(IndexInput.java:175)
>>        
>> org.apache.lucene.store.BufferedIndexInput.clone(BufferedIndexInput.java:128)
>>         org.apache.lucene.store.FSIndexInput.clone(FSDirectory.java:562)
>>        
>> org.apache.lucene.index.SegmentTermDocs.<init>(SegmentTermDocs.java:45)
>>        
>> org.apache.lucene.index.SegmentReader.termDocs(SegmentReader.java:333)
>>        
>> org.apache.lucene.index.MultiTermDocs.termDocs(MultiReader.java:416)
>>        
>> org.apache.lucene.index.MultiTermDocs.termDocs(MultiReader.java:409)
>>         org.apache.lucene.index.MultiTermDocs.next(MultiReader.java:361)
>>        
>> org.apache.lucene.index.ParallelReader$ParallelTermDocs.next(ParallelReader.java:353)
>>        
>> org.apache.lucene.search.FieldCacheImpl$3.createValue(FieldCacheImpl.java:173)
>>        
>> org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
>>        
>> org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:154)
>>        
>> org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:148)
>>        
>> org.apache.lucene.search.FieldSortedHitQueue.comparatorInt(FieldSortedHitQueue.java:204)
>>        
>> org.apache.lucene.search.FieldSortedHitQueue$1.createValue(FieldSortedHitQueue.java:175)
>>        
>> org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
>>        
>> org.apache.lucene.search.FieldSortedHitQueue.getCachedComparator(FieldSortedHitQueue.java:155)
>>        
>> org.apache.lucene.search.FieldSortedHitQueue.<init>(FieldSortedHitQueue.java:56)
>>        
>> org.apache.lucene.search.TopFieldDocCollector.<init>(TopFieldDocCollector.java:41)
... application stack

Another factor appears to be that this anomaly usually (maybe always)
happens just after a series of deleteAdd updates, i.e. just after a
series of deleting with the IndexReader, closing it to add a modified
version of that document with the IndexWriter, and then reopening the
IndexReader.  A query just after reopening the IndexReader is most
likely to trigger this issue.

I have not seen this problem on any other collecitons with the same
application, and so it may be specific to this collection or to its size.

Any thoughts or ideas would be appreciated.

Thanks,

Chuck


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message