lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Earwin Burrfoot (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-1658) Absorb NIOFSDirectory into FSDirectory
Date Tue, 26 May 2009 13:07:45 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712997#action_12712997
] 

Earwin Burrfoot commented on LUCENE-1658:
-----------------------------------------

bq. Wait, are you saying Win 64 bit has problems w/ MMapDir? (I thought you just said Win
32 bit, above).
I have no lucene experience with Win64, and so I extrapolated from 32bit (as I felt it was
more of a Windows issue than anything else). Would be nice if someone actually tries.
If it works, then the rule sounds like - MMap for all 64bit systems, NIO for 32bit non-win,
Simple for 32bit win.

bq. Is MMapDir fine w/ concurrency?
It's cool with it if you have enough memory (no frequent paging occurs). I'm not sure about
NIOFS vs MMap on memory-constrained systems, if you're competing for disk cache.

bq. Which way do you think "prime all bytes up front on open" should default?
This has a side-effect of pushing previous mmaps out of memory if you're memory-constrained.
Thus, under certain usage conditions (frequent merging, or something like that + low memory)
this feature theoretically could be a disadvantage.
For me it works well enough to be hardcoded to true :)

If anybody's interested, I can also repost an alternative for MMapD - MemoryMirroredD, which
wraps any given D and explicitly preloads files in non-chunked byte arrays. It's a bit faster
than MMapD and MemoryD (for reading), but has certain disadvantages, like stressing your GC
and throwing OOM on merges/optimize if you don't have enough heap (unlike MMapD that silently
swaps out).

> Absorb NIOFSDirectory into FSDirectory
> --------------------------------------
>
>                 Key: LUCENE-1658
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1658
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Store
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 2.9
>
>         Attachments: LUCENE-1658.patch, LUCENE-1658.patch
>
>
> I think whether one uses java.io.* vs java.nio.* or eventually
> java.nio2.*, or some other means, is an under-the-hood implementation
> detail of FSDirectory and doesn't merit a whole separate class.
> I think FSDirectory should be the core class one uses when one's index
> is in the filesystem.
> So, I'd like to deprecate NIOFSDirectory, absorbing it into
> FSDirectory, and add a setting "useNIO" to FSDirectory.  It should
> default to "true" for non-Windows OSs, because it gives far better
> concurrent performance on all platforms but Windows (due to known Sun
> JRE issue http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6265734).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message