lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (JIRA)" <>
Subject [jira] Commented: (LUCENE-1658) Absorb NIOFSDirectory into FSDirectory
Date Tue, 26 May 2009 11:07:46 GMT


Uwe Schindler commented on LUCENE-1658:

>From my experiences, MMap is always preferred, if you have 64 bit system. In this case,
Java maps the file into address space like a swap file and often used parts are in real memory.
So in my opinion, it is always preferable on 64 bit systems. If you have much RAM it is even
better (because caching).
On Linux/Solaris you can see the used address space with TOP. I have a webserver with an about
7 GB big index using nmap. It is reopened very often, so the used address space sometimes
goes up to >50 Gigabytes, but this is not a problem, as it is not real memory, its just
like a "swap file". The finalizer removes the mapped adress space fairly fast (dependent on
usage/closing of old segements).
The index in this directory is really fast, especially, if you have lots of real RAM, that
can be used for buffering. You can even load stored fields for thousands of documents very
fast, uninversion is also fast.
Concurrency is no problem as the mapped file is handled like RAM.

I always recommend users, who want to use lucene: Use 64 bit systems, -d64 JVM parameter (this
enables Java in 64 bit on Solaris), a lot of RAM and MMapDirectory.

> Absorb NIOFSDirectory into FSDirectory
> --------------------------------------
>                 Key: LUCENE-1658
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Store
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 2.9
>         Attachments: LUCENE-1658.patch, LUCENE-1658.patch
> I think whether one uses* vs java.nio.* or eventually
> java.nio2.*, or some other means, is an under-the-hood implementation
> detail of FSDirectory and doesn't merit a whole separate class.
> I think FSDirectory should be the core class one uses when one's index
> is in the filesystem.
> So, I'd like to deprecate NIOFSDirectory, absorbing it into
> FSDirectory, and add a setting "useNIO" to FSDirectory.  It should
> default to "true" for non-Windows OSs, because it gives far better
> concurrent performance on all platforms but Windows (due to known Sun
> JRE issue

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message