lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: java.lang.NegativeArraySizeException on searching using Integer.MAX_VALUE for number of hits
Date Wed, 02 Dec 2009 15:00:15 GMT
If you want to have all results, you do something wrong. :-)

Full text engines like lucene are made for returning only top-ranking
results. So if you use TopDocs results you must know before how many TopDocs
you want to have. Internally Lucene works with PriorityQueues that filter
the top ranking results.

If you want to have all results, you should not sort them by ranking, which
is not needed then). In this case, implement an own Collector and collect
the results for yourself into e.g. ArrayLists and so on (but they are
unsorted then).

Another possibility (if you really need the docs in relevance order) is to
run the search 2 times:
First only collect the top n=100 results. The TopDocs instance also returns
the max results. Using that number you can re-run the query to get all
ranked results - but this is generally a bad approach, because PQs get
slower for too many results, where order is not relevant.

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: Paul Taylor [mailto:paul_t100@fastmail.fm]
> Sent: Wednesday, December 02, 2009 2:34 PM
> To: java-user@lucene.apache.org
> Subject: java.lang.NegativeArraySizeException on searching using
> Integer.MAX_VALUE for number of hits
> 
> Hi, just upgraded my code to Lucene 3.0 and on one simple search I get
> the following stacktrace when I pass Integer.MAX_VALUE to the
> Searcher.search(Query query,int n) method, if I change the value to 1000
> it works okay.
> 
> 
> java.lang.NegativeArraySizeException
>     at
> org.apache.lucene.util.PriorityQueue.initialize(PriorityQueue.java:90)
>     at org.apache.lucene.search.HitQueue.<init>(HitQueue.java:67)
>     at
> org.apache.lucene.search.TopScoreDocCollector.<init>(TopScoreDocCollector.
> java:117)
>     at
> org.apache.lucene.search.TopScoreDocCollector.<init>(TopScoreDocCollector.
> java:37)
>     at
> org.apache.lucene.search.TopScoreDocCollector$InOrderTopScoreDocCollector.
> <init>(TopScoreDocCollector.java:42)
>     at
> org.apache.lucene.search.TopScoreDocCollector$InOrderTopScoreDocCollector.
> <init>(TopScoreDocCollector.java:40)
>     at
> org.apache.lucene.search.TopScoreDocCollector.create(TopScoreDocCollector.
> java:104)
>     at
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:167)
>     at org.apache.lucene.search.Searcher.search(Searcher.java:98)
>     at org.apache.lucene.search.Searcher.search(Searcher.java:108)
> 
> Now I know I should specify a max hits value, but I really want to
> return all matches , and regardless it shouldn't throw this exception,
> and in other search code which also has specifies Integer.MAX_VALUE the
> exception is not occurring.
> 
> 
> thanks Paul
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message