lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sdeck <scott.dec...@gmail.com>
Subject Re: Speed of grouped queries
Date Wed, 03 Jan 2007 19:37:14 GMT

Mucho thanks. I will look into these.
For more info, I have roughly 30000 documents now, and about 350,000 terms
When I do my queries I use the StandardAnalyzer with a whole slew of stop
words.
So, not sure if that might still be messing me up or not.
In the end, I may have to go with the prebuilt search indexes, which is no
fun.

I may just have to step through the lucene code to see if it is creating
large arrays somewhere that it doesn't need to, or could just cache. Not
sure. 

Will let you know more as I work on it tonight.
Sdeck


Steven Rowe wrote:
> 
> Hi Scott,
> 
> sdeck wrote:
>> I guess, any ideas why I would run out of heap memory by combining all of
>> those boolean queries together and then running the query? What is
>> happening
>> in the background that would make that occur? Is it storing something in
>> memory, like all of the common terms or something, to cause that to
>> occur?
> 
> Doug Cutting gives a formula for Lucene memory usage for queries here
> (from 2001):
> 
> <http://mail-archives.apache.org/mod_mbox/lucene-java-user/200111.mbox/%3c4BC270C6AB8AD411AD0B00B0D0493DF0D78A1C@mail.grandcentral.com%3e>
> 
> And some more info here about the term dictionary (from 2003):
> 
> <http://mail-archives.apache.org/mod_mbox/lucene-java-dev/200305.mbox/%3c3EC555CE.4080906@lucene.com%3e>
> 
> You might want to look at this thread, which has some discussion about
> omitting norms and the term dictionary (from 2005):
> 
> <http://www.nabble.com/Memory-Usage-tf523535.html>
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 
> 
> 

-- 
View this message in context: http://www.nabble.com/Speed-of-grouped-queries-tf2910499.html#a8147121
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message