lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: Memory usage
Date Wed, 26 May 2004 21:52:26 GMT
It is cached by the IndexReader and lives until the index reader is 
garbage collected.  50-70 searchable fields is a *lot*.  How many are 
analyzed text, and how many are simply keywords?

Doug

James Dunn wrote:
> Doug,
> 
> Thanks!  
> 
> I just asked a question regarding how to calculate the
> memory requirements for a search.  Does this memory
> only get used only during the search operation itself,
> or is it referenced by the Hits object or anything
> else after the actual search completes?
> 
> Thanks again,
> 
> Jim
> 
> 
> --- Doug Cutting <cutting@apache.org> wrote:
> 
>>James Dunn wrote:
>>
>>>Also I search across about 50 fields but I don't
>>
>>use
>>
>>>wildcard or range queries. 
>>
>>Lucene uses one byte of RAM per document per
>>searched field, to hold the 
>>normalization values.  So if you search a 10M
>>document collection with 
>>50 fields, then you'll end up using 500MB of RAM.
>>
>>If you're using unanalyzed fields, then an easy
>>workaround to reduce the 
>>number of fields is to combine many in a single
>>field.  So, instead of, 
>>e.g., using an "f1" field with value "abc", and an
>>"f2" field with value 
>>"efg", use a single field named "f" with values
>>"1_abc" and "2_efg".
>>
>>We could optimize this in Lucene.  If no values of
>>an indexed field are 
>>analyzed, then we could store no norms for the field
>>and hence read none 
>>into memory.  This wouldn't be too hard to
>>implement...
>>
>>Doug
>>
>>
> 
> ---------------------------------------------------------------------
> 
>>To unsubscribe, e-mail:
>>lucene-user-unsubscribe@jakarta.apache.org
>>For additional commands, e-mail:
>>lucene-user-help@jakarta.apache.org
>>
> 
> 
> 
> 
> 	
> 		
> __________________________________
> Do you Yahoo!?
> Friends.  Fun.  Try the all-new Yahoo! Messenger.
> http://messenger.yahoo.com/ 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message