lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ganesh" <emailg...@yahoo.co.in>
Subject Re: IndexDivisor
Date Thu, 03 Dec 2009 10:31:49 GMT
Thanks.

IndexDivisor means, When set to N, then one in every N*termIndexInterval terms in the index
is loaded into memory.

For example i am having 100,000 unique terms and termIndexInterval set to 5 then (100000 /
(5*128) ) terms (==X terms)will be loaded in to memory. 
If termIndexInterval is set to 500 then (100000 / (5*100*128) ) ==> (X/100) terms should
be loaded and the memory consmuped should be 100 times less than that of the first one. But
is not the case. The memory difference is very less. 

Could any one explain me the calculation of memory savings with respect to IndexDivisor. 

Regards
Ganesh


----- Original Message ----- 
From: "Danil Ε’ORIN" <torindan@gmail.com>
To: <java-user@lucene.apache.org>
Sent: Friday, November 27, 2009 4:00 PM
Subject: Re: IndexDivisor


Try to open with very large value (MAX_INT) it will load only first
term, and look up the rest from disk.

On Fri, Nov 27, 2009 at 12:24, Michael McCandless
<lucene@mikemccandless.com> wrote:
> If you are absolutely certain you won't be doing any lookups by term.
>
> The only use case I know of is internal, when Lucene's SegmentMerger
> is merging the segment with other segments. In this case, the merger
> does a linear iteration of all terms, and never a lookup by term, so
> we save CPU/RAM by not loading the terms index.
>
> Mike
>
> On Fri, Nov 27, 2009 at 5:16 AM, Ganesh <emailgane@yahoo.co.in> wrote:
>> Thanks,
>>
>> May i know the purpose of using negative value?
>>
>> Regards
>> Ganesh
>>
>> ----- Original Message -----
>> From: "Michael McCandless" <lucene@mikemccandless.com>
>> To: <java-user@lucene.apache.org>
>> Sent: Friday, November 27, 2009 3:17 PM
>> Subject: Re: IndexDivisor
>>
>>
>>> This is the expected behavior.
>>>
>>> If you intend to use the reader for searching, looking doc freq,
>>> deleting docs, etc, you must pass a non-negative value for
>>> indexDivisor.
>>>
>>> Mike
>>>
>>> On Fri, Nov 27, 2009 at 12:00 AM, Ganesh <emailgane@yahoo.co.in> wrote:
>>>> Hello all,
>>>>
>>>> I am using Lucene v2.9.1, If I open my reader with positive value for termInfosIndexDivisor
then the search works fine. If i set to -1, then search throws exception "terms index was
not loaded when this reader was created". Is this the intend behaviour?
>>>>
>>>> I thought by setting this value to -1, No memory will be used and the terms
will be fetched from the disk. Search might be slow but with very less memory consumption.
>>>>
>>>> Regards
>>>> Ganesh
>>>> Send instant messages to your online friends http://in.messenger.yahoo.com
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>> Send instant messages to your online friends http://in.messenger.yahoo.com
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Send instant messages to your online friends http://in.messenger.yahoo.com 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message