lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: java.lang.NegativeArraySizeException on searching using Integer.MAX_VALUE for number of hits
Date Sun, 06 Dec 2009 09:48:59 GMT
On Wed, Dec 2, 2009 at 10:18 AM, Paul Taylor <paul_t100@fastmail.fm> wrote:
> Uwe Schindler wrote:
>>
>> If you want to have all results, you do something wrong. :-)
>>
>> Full text engines like lucene are made for returning only top-ranking
>> results. So if you use TopDocs results you must know before how many
>> TopDocs
>> you want to have. Internally Lucene works with PriorityQueues that filter
>> the top ranking results.
>>
>> If you want to have all results, you should not sort them by ranking,
>> which
>> is not needed then). In this case, implement an own Collector and collect
>> the results for yourself into e.g. ArrayLists and so on (but they are
>> unsorted then).
>>
>> Another possibility (if you really need the docs in relevance order) is to
>> run the search 2 times:
>> First only collect the top n=100 results. The TopDocs instance also
>> returns
>> the max results. Using that number you can re-run the query to get all
>> ranked results - but this is generally a bad approach, because PQs get
>> slower for too many results, where order is not relevant.
>
> Yes thats all fine and is my long term approach, but  isn't there a bug
> here, (in the case that is  failing there is only one actual result ), you
> haven't really addressed why I am geting this exception

This is certainly confusing (that you hit a NegativeArraySizeException
on passing MAX_INT); it's because the PQ takes 1+ the size you pass in
(which wraps int around to -1).  I suppose we could bounds check that
in PQ, though if we do that very likely the next thing you hit is
OOME, which would be the better exception here.  I'll open an issue.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message