Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-dev@lucene.apache.org
Message-ID: <1174306853.1243334505661.JavaMail.jira@brutus>
Date: Tue, 26 May 2009 03:41:45 -0700 (PDT)
From: "Michael McCandless (JIRA)" <jira@apache.org>
To: java-dev@lucene.apache.org
Subject: [jira] Commented: (LUCENE-1658) Absorb NIOFSDirectory into
 FSDirectory
In-Reply-To: <1868008588.1243181445652.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/LUCENE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712952#action_12712952 ] 

Michael McCandless commented on LUCENE-1658:
--------------------------------------------

bq. Not completely sure why, but MMap failed for me on Win32 some time ago

I think on 32 bit JRE we shouldn't choose MMap in FSDir.open().

bq. It's a one-liner. I can make a patch, but while it works better than vanilla MMapD, I'm not yet sure it's the best approach. As for writing - okay, I'll tell you if it turns out to be anything good.

Which way do you think "prime all bytes up front on open" should default?

{quote}
There are a lot of variables. Just as you said, on 32bit systems you have to take care of address space. So, nice for small indexes, bad for big ones. But, mmap in Java cannot be explicitly closed, it is closed in finalizer, so even for a small index if you update really often, you can hit an OOM even though you have enough memory. MMapD failed for me on windows. But it is fast. It is absolutely, totally, uber-indespensible, if you have a 64bit system, fat index, memory to spare and require lots of fast searches.

So, you can probably enable it for non-Win 64bit?
{quote}

Wait, are you saying Win 64 bit has problems w/ MMapDir?  (I thought you just said Win 32 bit, above).

Is MMapDir fine w/ concurrency?  (I'd assume so).  So, if you had the choice (ie, you're running in env w/ plenty of virtual mem), MMapDir would be preferred over NIOFSDir?

On a 64 bit env that doesn't have that much RAM, MMapDir should fare no worse than NIOFSDir, right?  Ie both are competing for the same IO cache.  And so on a 64 bit JRE perhaps the default should always be MMAPDir.

> Absorb NIOFSDirectory into FSDirectory
> --------------------------------------
>
>                 Key: LUCENE-1658
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1658
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Store
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 2.9
>
>         Attachments: LUCENE-1658.patch, LUCENE-1658.patch
>
>
> I think whether one uses java.io.* vs java.nio.* or eventually
> java.nio2.*, or some other means, is an under-the-hood implementation
> detail of FSDirectory and doesn't merit a whole separate class.
> I think FSDirectory should be the core class one uses when one's index
> is in the filesystem.
> So, I'd like to deprecate NIOFSDirectory, absorbing it into
> FSDirectory, and add a setting "useNIO" to FSDirectory.  It should
> default to "true" for non-Windows OSs, because it gives far better
> concurrent performance on all platforms but Windows (due to known Sun
> JRE issue http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6265734).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org