lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-1658) Absorb NIOFSDirectory into FSDirectory
Date Sun, 31 May 2009 20:33:07 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-1658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714925#action_12714925
] 

Uwe Schindler commented on LUCENE-1658:
---------------------------------------

bq. I dug down and found that this test illegally opens SegmentReaders on files that IndexWriter
still has open for writing, and somehow this causes problems when using an MMapDir. I'll open
a separate issue and
put details there.

This is a NIO bug in windows, I assume. In Google I found a report at sun about this, too.
Mapped buffers from UNC-pathes have wrong bytes in their buffer.

By the way, the other failing tests are easy to fix:
Some tests check, if the IndexInput throws an IOException when reading past eof. When doing
this with a Byte buffer, the get() throws an BufferUnderflowException. It can be fixed like
this in MMapIndexInputs:
{code}
    public byte readByte() throws IOException {
      try {
        return buffer.get();
      } catch (BufferUnderflowException e) {
        throw new IOException("read past eof");
      }
    }
{code}

The other failures are harmless, but it would be good to fix them. I am working on that and
then test extensive.

The problem with not freeing the buffer can be fixed on windows using the bad hack with this
sun.misc.Cleaner class and PrivilegedAction (described in the bug report), but this depends
on Sun's internals and works only with Sun's JRE. And it may fail on some under-priviledged
environments like web containers.

But nevertheless, with this bad hack, my local version works now without any failing test
on Win32 using MMap.

> Absorb NIOFSDirectory into FSDirectory
> --------------------------------------
>
>                 Key: LUCENE-1658
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1658
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Store
>            Reporter: Michael McCandless
>            Assignee: Uwe Schindler
>            Priority: Minor
>             Fix For: 2.9
>
>         Attachments: LUCENE-1658-take2.patch, LUCENE-1658-take2.patch, LUCENE-1658-take3.patch,
LUCENE-1658-take3.patch, LUCENE-1658.patch, LUCENE-1658.patch, LUCENE-1658.patch
>
>
> I think whether one uses java.io.* vs java.nio.* or eventually
> java.nio2.*, or some other means, is an under-the-hood implementation
> detail of FSDirectory and doesn't merit a whole separate class.
> I think FSDirectory should be the core class one uses when one's index
> is in the filesystem.
> So, I'd like to deprecate NIOFSDirectory, absorbing it into
> FSDirectory, and add a setting "useNIO" to FSDirectory.  It should
> default to "true" for non-Windows OSs, because it gives far better
> concurrent performance on all platforms but Windows (due to known Sun
> JRE issue http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6265734).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message