lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simon Willnauer <simon.willna...@googlemail.com>
Subject Re: setTermInfosIndexDivisor
Date Thu, 25 Jun 2009 08:40:55 GMT
Hey there,

On Thu, Jun 25, 2009 at 9:10 AM, Ganesh<emailgane@yahoo.co.in> wrote:
> Hello all,
>
> I am using Lucene v2.4.1
>
> 1)
> I have build multiple indexes of total 30 million documents. My memory limit is 512 MB.
In this case i am getting frequently OOME. If i increased the memory limit to 1 GB / 1.5 GB
then it is working fine. My point is it will also will get exhausted when it reaches 60 /
90 million documents.
>
> I thought to use setTermInfosIndexDivisor, but even then the memory consumption is same.
This parameter has no effect. Whether this parameter should be set while building index? I
build the index using default value. After hitting OOME i am setting this.
I would be curious what you do to your index. do you have a lot of
pending deletes? do you call optimize frequently? In which situations
do you hit the OOM?
>
> Directory dir = FSDirectory.getDirectory(indexPath);
> IndexReader reader = IndexReader.open(dir, true);
> reader.setTermInfosIndexDivisor(5);
Loaded terms might not dominate your memory consumption in side
lucene. Again, you should provide more information of indexing, the
environment and the situation where the error occurs.

simon
>
> 2)
> IndexWriter.setTermIndexInterval  should be set while creating the index? If i build
the index with default value, After some time if i use this parameter, Whether there will
be some effect?
>
> Regards
> Ganesh
>
> Send instant messages to your online friends http://in.messenger.yahoo.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message