lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adrien Grand <jpou...@gmail.com>
Subject Re: "4.1 consuming more memory than 3.0.2 while Indexing"
Date Mon, 01 Apr 2013 12:08:44 GMT
On Mon, Apr 1, 2013 at 1:56 PM, Arun Kumar K <arunk786@gmail.com> wrote:
> Hi Guys,

Hi,

> I have been finding out the heap space requirement for indexing and
> searching with 3.0.2 vs 4.1 (with BlockPostings Format).
>
> I have a 2GB index with 1 million docs with around 42 fields with 40 fields
> being random strings.
>
> I have seen that memory for search has reduced by 5X with 4.1 (with
> BlockPostings Format) but the memory usage during indexing with 4.1 is
> around 800MB~1.7 GB whereas for 3.0.2 it is 300~600MB.
> But indexing time is almost same with both versions.

How did you measure memory usage ? Operating systems may report high
memory usage because MMapDirectory became the new default directory
implementation and memory-mapped files are taken into account in the
virtual memory of your processes. See
http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html
for more information.

-- 
Adrien

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message