lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-3178) Native MMapDir
Date Wed, 31 Aug 2011 18:12:09 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094740#comment-13094740
] 

Robert Muir commented on LUCENE-3178:
-------------------------------------

It might even be worth doing the mmap from NativePosixUtil, dealing with all the round-to-pagesize
etc that you need, and accessing it with sun.misc.Unsafe.

I did a little prototype a while back that stole the address from MappedByteBuffer and used
Unsafe for all ops with no bounds checks, and the performance improvements were pretty interesting
:) But the problem with that approach is you still can't FileChannel.map a file > Integer.MAX_VALUE,
meaning we have to handle all the stupidity of multiple mappings, but I think with a native
mmap call you could just map the whole thing and avoid this hassle...


> Native MMapDir
> --------------
>
>                 Key: LUCENE-3178
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3178
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: core/store
>            Reporter: Michael McCandless
>
> Spinoff from LUCENE-2793.
> Just like we will create native Dir impl (UnixDirectory) to pass the right OS level IO
flags depending on the IOContext, we could in theory do something similar with MMapDir.
> The problem is MMap is apparently quite hairy... and to pass the flags the native code
would need to invoke mmap (I think?), unlike UnixDir where the code "only" has to open the
file handle.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message