lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: Optimum Lucene’s MMapDirectory size on 64bit OS
Date Fri, 26 Sep 2014 22:10:25 GMT
Hi,

1 GiB is the maximum possible. The chunk size is only applicable for 32 bit JDKs because of
limited address space.

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: Gaurav gupta [mailto:gupta.gaurav0125@gmail.com]
> Sent: Friday, September 26, 2014 9:12 PM
> To: java-user@lucene.apache.org
> Subject: Optimum Lucene’s MMapDirectory size on 64bit OS
> 
> Hi,
> 
> As per the post "The Generics Policeman Blog
> <http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-
> 64bit.html>"
> , I am using the MMapDirectory for faster access(search and update
> operations ,mainly search) of Lucene 4.8.1 index files. I am contemplating
> what is the optimal maximum MMap value for my indexes. Is default i.e. 1
> GB
> (1 << 30) or higher?
> 
> I have 6 indexes of size varying from 65GB to 6 GB. Currently, I am using 1 GB
> as maxChunkSize : - *MMapDirectory(file, null, 1<<30) *for all indexes.
> But thinking of specifying the higher value for mmap (1 GB or higher) for
> bigger index having 65GB size and lower value (0.5 GB or less) for smaller
> index having size of 6 GB. Any suggestion/guidance on it ?
> 
> Also, per blog mmap is not a size of physical memory allocation but just a
> address space to map the index files. How to allocate more RAM to index
> files for better performance? We have enough RAM free out of 64 GB. Per
> blog, one should use the  mmap file, like - *MMapDirectory(file, null,
> 1<<30) *and let OS manage the physical memory allocation for the index
> files. Is my understanding correct ?
> 
> The Generics Policeman Blog
> <http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-
> 64bit.html> :-
> 
>    - *MMapDirectory does not consume additional memory and the size of
>    mapped index files is not limited by the physical memory available on your
>    server.* By mmap() files, we only reserve address space not memory!
>    Remember, address space on 64bit platforms is for free!
> 
> Thanks
> Gaurav


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message