lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (JIRA)" <>
Subject [jira] Updated: (LUCENE-1658) Absorb NIOFSDirectory into FSDirectory
Date Sun, 31 May 2009 21:50:07 GMT


Uwe Schindler updated LUCENE-1658:

    Attachment: LUCENE-1658-take3.patch

Atached is a patch that now works (at least on windows) for all three versions of FSDir. One
additional test was fixed to SimpleFSDir (because it assumesan BufferedIndexInput).

This patch contains the tweaked MMapDir that has the following features:
- Throws correct IOExceptions on read past EOF
- Is able to unmap the buffer when close is called (for cloned inputs nothing is done, this
is similar to other FSDirs). This unmapping is an "illegal and unsecure hack" according to
Sun, but I have seen other open source projects, that use it.

The problems with unmapping are: It may fail on specific non-Sun VMs and may hit SecurityExceptions.
If this happens, the close() call will throw an IOException. The good thing is: The virtual
memory usage is lower and with small indexes, the 32 bit VMs do not hit OOMs, if buffers are
not unmapped by GC early.

What do you think? Should we supply this "extended" MMapDirectory?
Earwin, did you try this, too?

All tests pass now :-) JUHU!

> Absorb NIOFSDirectory into FSDirectory
> --------------------------------------
>                 Key: LUCENE-1658
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Store
>            Reporter: Michael McCandless
>            Assignee: Uwe Schindler
>            Priority: Minor
>             Fix For: 2.9
>         Attachments: LUCENE-1658-take2.patch, LUCENE-1658-take2.patch, LUCENE-1658-take3.patch,
LUCENE-1658-take3.patch, LUCENE-1658-take3.patch, LUCENE-1658.patch, LUCENE-1658.patch, LUCENE-1658.patch
> I think whether one uses* vs java.nio.* or eventually
> java.nio2.*, or some other means, is an under-the-hood implementation
> detail of FSDirectory and doesn't merit a whole separate class.
> I think FSDirectory should be the core class one uses when one's index
> is in the filesystem.
> So, I'd like to deprecate NIOFSDirectory, absorbing it into
> FSDirectory, and add a setting "useNIO" to FSDirectory.  It should
> default to "true" for non-Windows OSs, because it gives far better
> concurrent performance on all platforms but Windows (due to known Sun
> JRE issue

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message