lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gaurav gupta <gupta.gaurav0...@gmail.com>
Subject Optimum Lucene’s MMapDirectory size on 64bit OS
Date Fri, 26 Sep 2014 19:11:32 GMT
Hi,

As per the post "The Generics Policeman Blog
<http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html>"
, I am using the MMapDirectory for faster access(search and update
operations ,mainly search) of Lucene 4.8.1 index files. I am contemplating
what is the optimal maximum MMap value for my indexes. Is default i.e. 1 GB
(1 << 30) or higher?

I have 6 indexes of size varying from 65GB to 6 GB. Currently, I am using 1
GB as maxChunkSize : - *MMapDirectory(file, null, 1<<30) *for all indexes.
But thinking of specifying the higher value for mmap (1 GB or higher) for
bigger index having 65GB size and lower value (0.5 GB or less) for smaller
index having size of 6 GB. Any suggestion/guidance on it ?

Also, per blog mmap is not a size of physical memory allocation but just a
address space to map the index files. How to allocate more RAM to index
files for better performance? We have enough RAM free out of 64 GB. Per
blog, one should use the  mmap file, like - *MMapDirectory(file, null,
1<<30) *and let OS manage the physical memory allocation for the index
files. Is my understanding correct ?

The Generics Policeman Blog
<http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html> :-

   - *MMapDirectory does not consume additional memory and the size of
   mapped index files is not limited by the physical memory available on your
   server.* By mmap() files, we only reserve address space not memory!
   Remember, address space on 64bit platforms is for free!

Thanks
Gaurav

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message