lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley" <yo...@apache.org>
Subject Re: 15 minute hang in IndexInput.clone() involving finalizers
Date Fri, 15 Dec 2006 16:24:03 GMT
That is odd, Chuck... looks like a JVM bug to me.

What do you think of the following patch?
http://issues.apache.org/jira/browse/LUCENE-750

-Yonik
http://incubator.apache.org/solr Solr, the open-source Lucene search server

On 12/15/06, Chuck Williams <chuck@manawiz.com> wrote:
> Hi All,
>
> I've had a bizarre anomaly arise in an application and am wondering if
> anybody has ever seen anything like this.  Certain queries, in not easy
> to reproduce cases, take 15-20 minutes to execute rather than a few
> seconds.  The same query is fast some times and anomalously slow
> others.  This is on a 1,000,000 document collection, but the problem
> seems independent of that.
>
> I took a bunch of thread dumps during the anomaly period.  There are 4
> threads executing the same query at the same time, and all 4 appear to
> spend almost the entire time trying to register finalizers as part of
> cloning an IndexInput within an application call to create a
> TopFieldDocCollector into which the results will be collected.  The
> actual search has not been launched yet, and will be reasonably quick
> when it is.  All 4 threads show this unchanging stack trace during the
> 15-20 minutes:
>
> >> == Thread Connection thread group.HttpConnection-26493-11 ===>
> >> java.lang.ref.Finalizer.add(Unknown Source)
> >>         java.lang.ref.Finalizer.<init>(Unknown Source)
> >>         java.lang.ref.Finalizer.register(Unknown Source)
> >>         java.lang.Object.clone(Native Method)
> >>         org.apache.lucene.store.IndexInput.clone(IndexInput.java:175)
> >>
> >> org.apache.lucene.store.BufferedIndexInput.clone(BufferedIndexInput.java:128)
> >>         org.apache.lucene.store.FSIndexInput.clone(FSDirectory.java:562)
> >>
> >> org.apache.lucene.index.SegmentTermDocs.<init>(SegmentTermDocs.java:45)
> >>
> >> org.apache.lucene.index.SegmentReader.termDocs(SegmentReader.java:333)
> >>
> >> org.apache.lucene.index.MultiTermDocs.termDocs(MultiReader.java:416)
> >>
> >> org.apache.lucene.index.MultiTermDocs.termDocs(MultiReader.java:409)
> >>         org.apache.lucene.index.MultiTermDocs.next(MultiReader.java:361)
> >>
> >> org.apache.lucene.index.ParallelReader$ParallelTermDocs.next(ParallelReader.java:353)
> >>
> >> org.apache.lucene.search.FieldCacheImpl$3.createValue(FieldCacheImpl.java:173)
> >>
> >> org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
> >>
> >> org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:154)
> >>
> >> org.apache.lucene.search.FieldCacheImpl.getInts(FieldCacheImpl.java:148)
> >>
> >> org.apache.lucene.search.FieldSortedHitQueue.comparatorInt(FieldSortedHitQueue.java:204)
> >>
> >> org.apache.lucene.search.FieldSortedHitQueue$1.createValue(FieldSortedHitQueue.java:175)
> >>
> >> org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72)
> >>
> >> org.apache.lucene.search.FieldSortedHitQueue.getCachedComparator(FieldSortedHitQueue.java:155)
> >>
> >> org.apache.lucene.search.FieldSortedHitQueue.<init>(FieldSortedHitQueue.java:56)
> >>
> >> org.apache.lucene.search.TopFieldDocCollector.<init>(TopFieldDocCollector.java:41)
> ... application stack
>
> Another factor appears to be that this anomaly usually (maybe always)
> happens just after a series of deleteAdd updates, i.e. just after a
> series of deleting with the IndexReader, closing it to add a modified
> version of that document with the IndexWriter, and then reopening the
> IndexReader.  A query just after reopening the IndexReader is most
> likely to trigger this issue.
>
> I have not seen this problem on any other collecitons with the same
> application, and so it may be specific to this collection or to its size.
>
> Any thoughts or ideas would be appreciated.
>
> Thanks,
>
> Chuck

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message