lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konrad Kolosowski" <konr...@ca.ibm.com>
Subject OutOfMemoryErrors searching with WildCardQueries
Date Thu, 12 Jun 2003 01:26:03 GMT
I need to proof an on-line system against Out Of Memory Errors, that some
times crash our system.  The system allows boolean searches with wild
cards.

It is not recommended to use WildCardQuery with wild card at the first
position.   Having wildcard at first position works for small number of
documents in the index but results in errors for a larger index (containing
3k of 1-2 pages docs).  If one types a query with many wild cards, close to
the beginning of terms, e.g.  a* OR b* OR ... OR z*, is not it going to
lead to the same problem?

If I impose a requirement that not first one but first 3 letters of a word
in a query cannot be a wild card.  Will it provide an additional safety and
reduce the memory consumption during search?  If it does than I think it
probably would not help when index contains large number of terms with
common prefix anyway.

If the index grows to hundred thousand documents, with users simultaneously
searching indexes for different locales, what is the best way to cup the
memory requirement?  Limiting number of terms, or number of terms
containing wild cards, or eliminating wild card searches altogether.

Thanks for explanation or any pointers.

Konrad Kolosowski


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message