lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@lucene.com>
Subject Re: OutOfMemoryErrors searching with WildCardQueries
Date Thu, 12 Jun 2003 18:28:48 GMT
Konrad Kolosowski wrote:
> If the index grows to hundred thousand documents, with users simultaneously
> searching indexes for different locales, what is the best way to cup the
> memory requirement?  Limiting number of terms, or number of terms
> containing wild cards, or eliminating wild card searches altogether.

This was discussed recently on lucene-dev@jakarta.apache.org in a thread 
whose subject contains "too many hits - OutOfMemoryError".

I checked in a patch which limits the number of terms that a wildcard is 
permitted to expand into.  The default is 1000.  If a term expands to 
more than that then an exception is thrown.  Each term that a wildcard 
expands into requires around 2kB.  So this limits each wildcarded query 
term to 2MB.  If you have queries with large numbers of wildcarded terms 
then you might consider also limiting that.

This patch is in the latest version of Lucene in CVS, but not yet in a 
release.

Doug


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message