Return-Path: Delivered-To: apmail-jakarta-lucene-user-archive@apache.org Received: (qmail 50266 invoked from network); 12 Jun 2003 22:53:09 -0000 Received: from exchange.sun.com (192.18.33.10) by daedalus.apache.org with SMTP; 12 Jun 2003 22:53:09 -0000 Received: (qmail 11899 invoked by uid 97); 12 Jun 2003 22:55:28 -0000 Delivered-To: qmlist-jakarta-archive-lucene-user@nagoya.betaversion.org Received: (qmail 11891 invoked from network); 12 Jun 2003 22:55:28 -0000 Received: from daedalus.apache.org (HELO apache.org) (208.185.179.12) by nagoya.betaversion.org with SMTP; 12 Jun 2003 22:55:28 -0000 Received: (qmail 49910 invoked by uid 500); 12 Jun 2003 22:53:05 -0000 Mailing-List: contact lucene-user-help@jakarta.apache.org; run by ezmlm Precedence: bulk List-Unsubscribe: List-Subscribe: List-Help: List-Post: List-Id: "Lucene Users List" Reply-To: "Lucene Users List" Delivered-To: mailing list lucene-user@jakarta.apache.org Received: (qmail 49883 invoked from network); 12 Jun 2003 22:53:05 -0000 Received: from e5.ny.us.ibm.com (32.97.182.105) by daedalus.apache.org with SMTP; 12 Jun 2003 22:53:05 -0000 Received: from northrelay02.pok.ibm.com (northrelay02.pok.ibm.com [9.56.224.150]) by e5.ny.us.ibm.com (8.12.9/8.12.2) with ESMTP id h5CMr9td199592 for ; Thu, 12 Jun 2003 18:53:09 -0400 Received: from d25ml01.torolab.ibm.com (d01av02.pok.ibm.com [9.56.224.216]) by northrelay02.pok.ibm.com (8.12.9/NCO/VER6.5) with ESMTP id h5CMr7Ev216332 for ; Thu, 12 Jun 2003 18:53:08 -0400 In-Reply-To: <3EE8C660.9030808@lucene.com> Subject: Re: OutOfMemoryErrors searching with WildCardQueries To: "Lucene Users List" X-Mailer: Lotus Notes Release 6.0.1CF1 March 04, 2003 Message-ID: From: "Konrad Kolosowski" Date: Thu, 12 Jun 2003 18:53:03 -0400 X-MIMETrack: Serialize by Router on D25ML01/25/M/IBM(Release 5.0.9a |January 7, 2002) at 06/12/2003 06:53:08 PM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N After Dave Kor put me on track, I thought I will need to dive into hacking Lucene on my own, but having the fix already in the repository is great. Thank you Doug. I assume the fix will be picked up by 1.3 release. Is there an expected time frame for 1.3 Final build? Thanks. Konrad Kolosowski Doug Cutting om> cc: Subject: Re: OutOfMemoryErrors searching with WildCardQueries 06/12/2003 02:28 PM Please respond to "Lucene Users List" Konrad Kolosowski wrote: > If the index grows to hundred thousand documents, with users simultaneously > searching indexes for different locales, what is the best way to cup the > memory requirement? Limiting number of terms, or number of terms > containing wild cards, or eliminating wild card searches altogether. This was discussed recently on lucene-dev@jakarta.apache.org in a thread whose subject contains "too many hits - OutOfMemoryError". I checked in a patch which limits the number of terms that a wildcard is permitted to expand into. The default is 1000. If a term expands to more than that then an exception is thrown. Each term that a wildcard expands into requires around 2kB. So this limits each wildcarded query term to 2MB. If you have queries with large numbers of wildcarded terms then you might consider also limiting that. This patch is in the latest version of Lucene in CVS, but not yet in a release. Doug --------------------------------------------------------------------- To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org For additional commands, e-mail: lucene-user-help@jakarta.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org For additional commands, e-mail: lucene-user-help@jakarta.apache.org