lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-1658) Absorb NIOFSDirectory into FSDirectory
Date Mon, 25 May 2009 18:53:46 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712773#action_12712773
] 

Michael McCandless commented on LUCENE-1658:
--------------------------------------------

bq. In the end, I'm unsure if it's a good idea to fold anything into FSD at all.

OK, you convinced me: let's keep separate classes.

But I'd like to split the current FSDir into an abstract FSDir and a
subclass SimpleFSDir.  Then, we have three subclasses of FSDir
(Simple, NIO, MMap).  And strengthen the javadocs so SimpleFSDir's
concurrency limitations are clear.

bq. I assume your initial aim is to provide users with best impl for current platform without
making them think. What about static factory that chooses between several impls?

Right, that's my goal: there should be a single obvious way to ask for
a Directory that uses the file system, and that way should have good
defaults.

FSDirectory is unfortunately the obvious way now, but it's usually a
poor choice.

bq. For example, my version of MMapD uses MappedByteBuffer.load() on creating MMapIndexInput.

Is this something you could share?  Sounds useful!  Likewise if you
extend MMapDir for writing...

Does anyone have a strong sense of when MMapDir is / is not an
appropriate choice?  I've seen some users report good performance, eg:

  http://mail-archives.apache.org/mod_mbox/lucene-java-user/200603.mbox/%3C20060313025744.18818.qmail@station174.com%3E

MMap eats into the virtual memory of the process, so on 32 bit JRE
that obviously a very real concern.

bq. As the separate public constructors and deprecation of FSDirectory.open() is new in 2.9

Right we are free to change things still (hmm: a nice side effect of
NOT releasing very often; incentives aren't quite right...).

But the old API was FSDirectory.getDirectory.

I think we should add a static FSDirectory.create() that returns a
good default impl given your OS.


> Absorb NIOFSDirectory into FSDirectory
> --------------------------------------
>
>                 Key: LUCENE-1658
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1658
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Store
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 2.9
>
>         Attachments: LUCENE-1658.patch
>
>
> I think whether one uses java.io.* vs java.nio.* or eventually
> java.nio2.*, or some other means, is an under-the-hood implementation
> detail of FSDirectory and doesn't merit a whole separate class.
> I think FSDirectory should be the core class one uses when one's index
> is in the filesystem.
> So, I'd like to deprecate NIOFSDirectory, absorbing it into
> FSDirectory, and add a setting "useNIO" to FSDirectory.  It should
> default to "true" for non-Windows OSs, because it gives far better
> concurrent performance on all platforms but Windows (due to known Sun
> JRE issue http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6265734).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message