lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: SorterTemplate.quickSort causes StackOverflowError
Date Thu, 28 Apr 2011 21:36:13 GMT
Hi Otis,

Can you reproduce this somehow and send test code? I could look into it. I
don't expect the error in the quicksort algorithm itself as this one is used
e.g. BytesRefHash / TermsHash, if there is a bug we would have seen it long
time ago.

I have not seen this before, but I suspect a problem in this very strange
comparator in MemoryIndex (which is very broken, if you look at its code -
it can compare Strings with Map.Entry and so on, brrrr), maybe the
comparator is not stable? In this case, quicksort can easily loop endless
and stack overflow. In Lucene 3.0 this used stock java sort (which is
mergesort), maybe replace the ArrayUtils.quickSort my ArrayUtils.mergeSort()
and see if problem is still there?

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com]
> Sent: Thursday, April 28, 2011 11:17 PM
> To: java-user@lucene.apache.org
> Subject: SorterTemplate.quickSort causes StackOverflowError
> 
> Hi,
> 
> I'm looking at some code that uses MemoryIndex (Lucene 3.1) and that's
> exhibiting a strange behaviour - it slows down over time.
> The MemoryIndex contains 1 doc, of course, and executes a set of a few
> thousand queries against it.  The set of queries does not change - the
same
> set of queries gets executed on all incoming documents.
> This code runs very quickly..... in the beginning.   But with time is gets
> slower and slower.... and slower..... and then I get this:
> 
> 4/28/11 10:32:52 PM (S) SolrException.log : java.lang.StackOverflowError
>     at
> org.apache.lucene.util.SorterTemplate.quickSort(SorterTemplate.java:104)
>     at
> org.apache.lucene.util.SorterTemplate.quickSort(SorterTemplate.java:104)
>     at
> org.apache.lucene.util.SorterTemplate.quickSort(SorterTemplate.java:104)
> 
> I haven't profiled this code yet (remote server, firewall in between,
can't use
> YourKit...), but does the above look familiar to anyone?
> I've looked at the code and obviously there is the recursive call that's
> problematic here - it looks like the recursion just gets deeper and deeper
and
> "gets stuck", eventually getting too deep for the JVM's taste.
> 
> Thanks,
> Otis
> ----
> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem
> search :: http://search-lucene.com/
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message