lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: Running OutOfMemory while optimizing and searching
Date Thu, 01 Jul 2004 19:36:25 GMT
 > What do your queries look like?  The memory required
 > for a query can be computed by the following equation:
 >
 > 1 Byte * Number of fields in your query * Number of
 > docs in your index
 >
 > So if your query searches on all 50 fields of your 3.5
 > Million document index then each search would take
 > about 175MB.  If your 3-4 searches run concurrently
 > then that's about 525MB to 700MB chewed up at once.

That's not quite right.  If you use the same IndexSearcher (or 
IndexReader) for all of the searches, then only 175MB are used.  The 
arrays in question (the norms) are read-only and can be shared by all 
searches.

In general, the amount of memory required is:

1 byte * Number of searchable fields in your index * Number of docs in 
your index

plus

1k bytes * number of terms in query

plus

1k bytes * number of phrase terms in query

The latter are for i/o buffers.  There are a few other things, but these 
are the major ones.

Doug


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message