Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of uwe@thetaphi.de designates
 188.138.97.18 as permitted sender)
From: "Uwe Schindler" <uwe@thetaphi.de>
To: <java-user@lucene.apache.org>
References: 
 <CALhtWkfb6tjHFiRxLdM5GzOnZYwkdGYxmq8pjiD_167m9JM5DA@mail.gmail.com>
In-Reply-To: 
 <CALhtWkfb6tjHFiRxLdM5GzOnZYwkdGYxmq8pjiD_167m9JM5DA@mail.gmail.com>
Subject: RE: Counting all the hits with parallel searching
Date: Sun, 19 Feb 2012 15:53:54 +0100
Message-ID: <002b01ccef16$4cd41c30$e67c5490$@thetaphi.de>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Thread-Index: AQFBy97GOkOTyVVLOohiblWdUIeqL5dak70Q
Content-Language: de

By passing Integer.MAX_VALUE you are requesting Lucene to allocate a =
priority queue for collecting results with that size, this OOMs. With =
Lucene if you are using TopDocs, the idea is to only get a limited =
amount of Top-Ranking documents to display search results. The user is =
not interested in the 2 million's result page, so pass a small number of =
top hits.

To simply count all hits like you seem to do, there is a separate =
collector available: http://goo.gl/XsPVR

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: Benson Margulies [mailto:bimargulies@gmail.com]
> Sent: Sunday, February 19, 2012 3:22 PM
> To: java-user@lucene.apache.org
> Subject: Counting all the hits with parallel searching
>=20
> If I have a lot of segments, and an executor service in my searcher, =
the
> following runs out of memory instantly, building giant heaps. Is there =
another
> way to express this? Should I file a JIRA that the parallel code =
should have some
> graceful behavior?
>=20
> int longestMentionFreq =3D searcher.search(longestMentionQuery, =
filter,
> Integer.MAX_VALUE).totalHits + 1;
>=20
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org