lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kevin A. Burton" <bur...@newsmonster.org>
Subject Re: Changing INDEX_INTERVAL to allow smaller memory footprint?
Date Mon, 14 Feb 2005 20:24:09 GMT
Kevin A. Burton wrote:

> I started a thread about a week ago about the memory footprint of 
> opening up a lucene index.

Ug... you know I'm sorry. Doug responded to this but I didn't see his 
followup.  I'll try this change this week and see what happens.

> You can increase TermInfosWriter.indexInterval.  You'll need to 
> re-write the .tii file for this to take effect.  The simplest way to 
> do this is to use IndexWriter.addIndexes(), adding your index to a 
> new, empty, directory.  This will of course take a while for a 60GB 
> index...
>
> Doubling TermInfosWriter.indexInterval should half the Term memory 
> usage and double the time required to look up terms in the 
> dictionary.  With an index this large the the latter is probably not 
> an issue, since processing term frequency and proximity data probably 
> overwhelmingly dominate search performance.
>
> Perhaps we should make this public by adding an IndexWriter method?
>
> Also, you can list the size of your .tii file by using the main() from 
> CompoundFileReader.

Kevin

-- 

Use Rojo (RSS/Atom aggregator).  Visit http://rojo.com. Ask me for an 
invite!  Also see irc.freenode.net #rojo if you want to chat.

Rojo is Hiring! - http://www.rojonetworks.com/JobsAtRojo.html

If you're interested in RSS, Weblogs, Social Networking, etc... then you 
should work for Rojo!  If you recommend someone and we hire them you'll 
get a free iPod!
    
Kevin A. Burton, Location - San Francisco, CA
       AIM/YIM - sfburtonator,  Web - http://peerfear.org/
GPG fingerprint: 5FB2 F3E2 760E 70A8 6174 D393 E84D 8D04 99F1 4412


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-dev-help@jakarta.apache.org


Mime
View raw message