lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cory Albright <albri...@mt.sri.com>
Subject Re: too many hits - OutOfMemoryError
Date Wed, 28 May 2003 16:16:30 GMT
Thanks for the help!  Yes, it works fine without a wildcard search, which
I believe at this point will be ok for our app.

Thanks again,

Cory

At 11:50 AM 5/28/2003 -0400, you wrote:

>Cory,
>
>When performing wildcard queries, the bulk of the memory is used during
>wildcard term expansion.  The memory requirement is proportional to the
>number of matching terms, not the number of hits.
>
>You should make sure you are using the latest Lucene.  There was a fix in
>1.3 to reduce the memory requirements of all all queries.
>
>But, wildcard queries that expand to many terms are allways going to be
>memory intensive in Lucene.  We ran into this problem and decided to put a
>check on the number of expanded terms and abort the query if the number got
>too high.  If you're ambitious, you could modify the Lucene source to
>serialize the query process for queries with a large number of terms, but
>that would be a bit of work.  If you absolutely require these huge wildcard
>queries, then you may have to look into it, though.
>
>Non-wildcard queries that return a large result set should not be a memory
>problem, though.
>
>Dave
>
>
>
>
> 
>
>                       Cory 
> Albright
>                       <albright@mt.sri.        To:       "Lucene Users 
> List"
>                       com> 
> <lucene-user@jakarta.apache.org>
>                                                cc: 
>
>                       05/28/03 11:16 AM        Subject:  Re: too many 
> hits - OutOfMemoryError
>                       Please respond 
> to
>                       "Lucene 
> Users
>                       List" 
>
> 
>
> 
>
>
>
>
>
>Yes. Is that the problem?
>
>At 05:13 PM 5/28/2003 +0200, you wrote:
> > > When I search with a query I know will hit most of the 1.8 million
> > > records, the "collect" print
> > > does not even print, it eats up the 700+MB I allocated and then
> > > throws an OutOfMemoryError.
> >
> >Are you using wildcard queries?
> >
> >--
> >Eric Jain
> >
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> >For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
>For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>
>
>
>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
>For additional commands, e-mail: lucene-user-help@jakarta.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message