lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eks Dev (JIRA)" <>
Subject [jira] Commented: (LUCENE-1743) MMapDirectory should only mmap large files, small files should be opened using SimpleFS/NIOFS
Date Tue, 14 Jul 2009 19:55:14 GMT


Eks Dev commented on LUCENE-1743:

indeed! obvious idea, 

the only thing I do not like with it is making these hidden, deceptive decisions "I said I
want MMapDirectory and someone else decided something else for me"... it does not matter if
we have conses here now, it may change tomorrow 

probably better way would be to turbo charge FileSwitchDirectory with sexy parametrization
MMapDirectory <- F(fileExtension, minSize, maxSize) // If <fileExtension> and file
size less than <maxSize> and greater than <minSize> than open file with MMapDirectory...
than go on on next rule... (can be designed upside down as well... changes nothing in idea)

the same for RAMDir, NIO, FS... 

With this, we can make UwesBestOfMMapDirectoryFor32BitOSs (your proposal here) or 
HighlyConcurentForWindows64WithTermDictionaryInRamAndStoredFieldsOnDiskDirectory just for
me :) 

So the most of the end users take some smart defaults we provide in core, and freaks (Expert
users in official lingo :) have their job easy, just to configure TurboChargedFileSwitchDirectory

Should be easy to come up with clean design for these "Concrete Directory selection rules"
by keeping concrete Directories "pure"

Cheers, Eks 

> MMapDirectory should only mmap large files, small files should be opened using SimpleFS/NIOFS
> ---------------------------------------------------------------------------------------------
>                 Key: LUCENE-1743
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Store
>    Affects Versions: 2.9
>            Reporter: Uwe Schindler
>            Assignee: Uwe Schindler
>             Fix For: 3.1
> This is a followup to LUCENE-1741:
> Javadocs state (in FileChannel#map): "For most operating systems, mapping a file into
memory is more expensive than reading or writing a few tens of kilobytes of data via the usual
read and write methods. From the standpoint of performance it is generally only worth mapping
relatively large files into memory."
> MMapDirectory should get a user-configureable size parameter that is a lower limit for
mmapping files. All files with a size<limit should be opened using a conventional IndexInput
from SimpleFS or NIO (another configuration option for the fallback?).

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message