lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Francesco Bellomi" <fbell...@libero.it>
Subject Directory implementation using NIO
Date Sun, 06 Jul 2003 20:49:59 GMT
Hi,

I developed a Directory implementation that accesses an index stored on the
filesystem using memory-mapped files (as provided by the NIO API, introduced
in Java 1.4).

You can download the complied jar and the source from here:
www.fran.it/lucene-NIO.zip

Basically there are 3 new classes: NIODirectory, NIOInputStream and
NIOOutputStream. They are heavily based on FSDirectory, FSInputStream and
FSOutputStream.

NIOInputStram provides memory-mapped access to files. It does not rely on
Lucene InputStream's caching feature, since direct access to the
memory-mapped file should be faster. Also, cloned stream with independent
positions are implemented using NIO buffer duplication (a buffer duplicate
holds the same content but has its own position), and so the implementation
logic is much simpler than FSInputStream's.

Some methods of Directory have been overridden to replace the caching
feature. Some of then were final in Directory, so I have used a slightly
modified version of Directory.java (BTW, I wonder why so many methods in
Lucene are made final...)

These classes only works with the recently released Java 1.4.2. This is due
to the fact that buffers connected with memory-mapped files could not be
programmatically unmapped in previous releases, (they were unmapped only
through finalization) and actively mapped files cannot be deleted. These
issue are partially resolved with 1.4.2.

NIOOutputStream is the same as FSOutputStream; I don't know any way to take
advantege of NIO for writing indexes (memory mapped buffers have a static
size, so they are not useful if your file is growing).

I don't have a benchmarking suite for Lucene, so I can't accurately evaluate
the speed of this implementation. I tested it on a small application I am
developing and it seems to work well, but I think my test are not
significative. Of course only the searching feature is expected to be
faster, since the index writing is unchanged.

Francesco

-
Francesco Bellomi
"Use truth to show illusion,
and illusion to show truth."



---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message