jackrabbit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jukka Zitting" <jukka.zitt...@gmail.com>
Subject Re: NGP: Prototyping
Date Thu, 15 Nov 2007 13:36:58 GMT

On Nov 15, 2007 11:07 AM, Thomas Mueller <thomas.tom.mueller@gmail.com> wrote:
> A few remarks about the current NGP prototype:
> java.nio.ByteBuffer (FileJournal / MediumRecord)
> Just curious, did you run some tests, is it faster than using
> RandomAccessFile.read?

I'm planning to... It would be really nice to map the entire journal
file (or the part that already exists when a repository is started)
into memory for direct access. In theory this should reduce the amount
of copying taking place, but it's still unclear whether that happens
in practice.

> FileChannel.map
> My experience with memory mapped files is: it is fast, but
> problematic. The biggest problem (for me) is: there is no way to unmap
> a file. That means it is not possible to truncate, rename, or delete a
> file that was once mapped. See:
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4724038
> Journal:
> "A record is never modified or removed once it has been added to a
> journal.". I agree, if there is a mechanism to removing old journal
> files.

The way I see it, during normal operation we only need to be able to
append data to the file, so this shouldn't be a problem. At some
points the journal file needs to be vacuumed to save space, but that
operation would typically produce a new file that's then used to
replace the old journal. I'm not yet sure how often and at which
points such vacuuming needs to take place.

Also, I have an experimental C version of the journal and tree code.
The idea behind that is possibly to implement a mod_dav module to
enable direct WebDAV access to the tree structure stored in the
journal. The C implementation could also eventually become a highly
optimized JNI backend for an NGP implementation, in which case we'd
also have much more control over memory mapping (of course at the
expense of operating system dependencies).


Jukka Zitting

View raw message