Mailing-List: contact lucene-user-help@jakarta.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Lucene Users List" <lucene-user@jakarta.apache.org>
Message-ID: <20040526214140.62527.qmail@web41101.mail.yahoo.com>
Date: Wed, 26 May 2004 14:41:40 -0700 (PDT)
From: James Dunn <james_h_dunn@yahoo.com>
Subject: Re: Memory usage
To: Lucene Users List <lucene-user@jakarta.apache.org>
In-Reply-To: <40B50C78.4020200@apache.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

Doug,

Thanks!  

I just asked a question regarding how to calculate the
memory requirements for a search.  Does this memory
only get used only during the search operation itself,
or is it referenced by the Hits object or anything
else after the actual search completes?

Thanks again,

Jim


--- Doug Cutting <cutting@apache.org> wrote:
> James Dunn wrote:
> > Also I search across about 50 fields but I don't
> use
> > wildcard or range queries. 
> 
> Lucene uses one byte of RAM per document per
> searched field, to hold the 
> normalization values.  So if you search a 10M
> document collection with 
> 50 fields, then you'll end up using 500MB of RAM.
> 
> If you're using unanalyzed fields, then an easy
> workaround to reduce the 
> number of fields is to combine many in a single
> field.  So, instead of, 
> e.g., using an "f1" field with value "abc", and an
> "f2" field with value 
> "efg", use a single field named "f" with values
> "1_abc" and "2_efg".
> 
> We could optimize this in Lucene.  If no values of
> an indexed field are 
> analyzed, then we could store no norms for the field
> and hence read none 
> into memory.  This wouldn't be too hard to
> implement...
> 
> Doug
> 
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> lucene-user-help@jakarta.apache.org
> 


__________________________________
Do you Yahoo!?
Friends.  Fun.  Try the all-new Yahoo! Messenger.
http://messenger.yahoo.com/ 

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org