lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Danil ŢORIN <torin...@gmail.com>
Subject Re: IndexDivisor
Date Thu, 03 Dec 2009 15:05:08 GMT
Run System.gc() exactly before measuring memory usage.

On sun jvm it will FORCE gc (unless DisableExplicitGC is used).


On Thu, Dec 3, 2009 at 16:30, Ganesh <emailgane@yahoo.co.in> wrote:
> Thanks mike.
>
> I am opening the reader and warming it up and then calculating the memory consumed.
> long usedMemory   = runtime.totalMemory() - runtime.freeMemory();
>
> Regards
> Ganesh
>
> ----- Original Message -----
> From: "Michael McCandless" <lucene@mikemccandless.com>
> To: <java-user@lucene.apache.org>
> Sent: Thursday, December 03, 2009 6:22 PM
> Subject: Re: IndexDivisor
>
>
> On Thu, Dec 3, 2009 at 7:15 AM, Ganesh <emailgane@yahoo.co.in> wrote:
>
>> Below are my stats
>> IndexDivisor Memory
>> -1 7 MB
>> 1 486 MB
>> 100 180 MB
>> 1000 176 MB.
>
> Do you simply create the IndexWriter & IndexReader, but do no
> searching/indexing?
>
> How are you measuring RAM?  You should use a RAM profiler (eg YourKit)
> to get the "real" usage, not counting garbage that's yet to be
> collected.
>
> My guess is the shear process of Lucene scanning your terms index
> creates lots of garbage and you're measuring the RAM consumed by that
> garbage.  The garbage should be harmless (it'll eventually get GCd &
> reused).
>
>> 1. Whether term vector will consume RAM. I think it should only consume disk space.
>
> No RAM is consumed.
>
>> 2. By setting RAM buffer for IW, Does it will allocate memory Or based on the usage
the memory will be allocated / increased.
>
> Only a little RAM is allocated up front, I think.  It's only as you
> start indexing docs that the big RAM is really allocated.  After a
> flush the RAM is then reused (ie, not freed) for subsequent segments.
>
> Mike
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
> Send instant messages to your online friends http://in.messenger.yahoo.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message